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4 The epistemic status of scientific 
visualisations 


As the considerations in the previous chapter made clear, visual representations 
are, without doubt, part of many epistemic processes in contemporary 
science. Scientists present diagrams in their publications and talks to com- 
municate their research results. They investigate computer-generated images 
as substitutes for research objects. Drawings in textbooks are used to educate 
novices, to introduce them to a new field of knowledge and so on. Moreover, 
it was pointed out that in quite a few instances images might also be used 
for non-epistemic purposes, for example to gain the attention of a particular 
audience. 

The preceding contextual analysis helped to clarify and, to some degree, to 
systematise these diverse functional roles played by visualisations in science. 
However, although the functions of visual representations in epistemic 
contexts have been rendered identifiable, it might be argued that this is only a 
descriptive result. That is, merely noting and assessing scientific activities does 
not imply that these processes are also the best way to achieve the epistemic 
aims aspired to. Perhaps scientists are wrong in their decision to rely on visual 
representations to such a degree in their epistemic processes. Perhaps they 
should make use of other modes of representations in those instances instead. 

These considerations correspond to the two perspectives inherent to the 
philosophy of science. Thomas S. Kuhn advises paying close attention to what 
is really going on in the sciences, how practices and theoretical assumptions 
have developed over time, and in what sense social processes have played a 
role in epistemic contexts. This is the descriptive task of the philosophy of 
science, the component that is supposed to connect philosophers’ hypotheses 
and theories to real world activities in science. Considering ideal situations 
might be interesting and sometimes helpful, but clinging to an unrealisable 
ideal can also turn out to be destructive in the long run — for example, by 
wrongly eliminating established working practices. Yet Kuhn’s advice does 
not mean that philosophers of science have to confine themselves to a mere 
unreflected recording of activities. On the contrary, there is also a second, 
normative part of their work. That is, not only are philosophers supposed to 
record, but also to analyse and evaluate epistemic practices observed in the 
scientists’ strivings for knowledge. These analyses might result in regarding 
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the prevalent practices as being defective and requiring improvements in 
some way. Thus, philosophers are also asked to make suggestions on how to 
improve those practices from an epistemological point of view.! Hans Poser, 
for instance, discusses these two aspects of the philosophy of science (see Poser 
2001). He points out that, among the different disciplines that are concerned 
with science on a meta-level — such as the history of science, the sociology of 
science, etc. — the philosophy of science is unique in that it deals with descrip- 
tive as well as with normative questions about its subject (see ibid., 15ff., 36). 

At least, parts of our analysis of the epistemic functions of scientific 
images are in this sense a descriptive endeavour. Now I also want to add 
a normative dimension. I will investigate the question of whether scientists 
are right to use visualisations in the way they do. Leaving non-epistemic 
purposes of image practices aside, issues such as the following will have to 
be discussed: can people gain knowledge from visual representations? If so, 
what kind of knowledge do they acquire by those means? How does the pro- 
cess of information transmission work with respect to visualisations? Are 
there crucial differences in information transmission via numerical or verbal 
representations? Is there a kind of hierarchy involved when comparing these 
three modes of representation from an epistemological point of view? Such a 
comparative task highlights what is at stake here, namely the precise epistemic 
status that visual representations possess in comparison to other represen- 
tational means in epistemically relevant contexts. It has to be added that, as 
the functional roles of visual representations are diverse in science, it seems 
rational to assume that there is not one exclusive status to be ascribed to these 
representational means, but various ones. 

In this chapter, I will focus on the explanatory context of science, the 
context of information transmission and the role of visual representations 
therein. More precisely, I am interested in the two following questions: are 
visual representations a suitable means in this epistemic context at all, and are 
there epistemic purposes that visual representations are particularly suitable 
to serve in comparison with competing modes of representation? 

To find answers to these questions, I shall proceed as follows: firstly, an 
examination is needed concerning the basic philosophical problem that 
underlies the discussion of the epistemic status of scientific images. The 
question is why philosophers are concerned about visual representations 
in epistemic processes at all? What do they think is wrong with scientific 
practices making use of images? Is there anything wrong at all? At this point, 
there will be a return to Perini’s considerations, mentioned at the beginning of 
this book. The apparent tension, pointed out by her, that appears when com- 
paring actual scientific practices and philosophical reflections about them (see 
Perini 2005c, 913f.) will be discussed, as well as the nature of arguments, and 
it will also be explained why philosophers think that visual representations 
are not suitable means to be used in argumentation. This critical attitude will 
be contrasted nonetheless with some suggestions on how to conceive visual 
arguments. 
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These considerations will direct an investigation into the cognitive content 
of visual representations, which will form the second part of the analysis. On 
the one hand, that information presented in different representational modes 
can, at least partly, be translated from one mode to another speaks in favour 
of the thesis that visual representations can contain a cognitively access- 
ible content. On the other, as already mentioned, some philosophers and 
semioticians (see e.g. Eco 1994, ch. 7) object that communicative acts based 
on visual representations face serious limitations of expression. I will con- 
trast these rather critical stances towards the epistemic capabilities of visual 
representations with the more optimistic assessments put forward by Kulvick1 
as well as by Kitcher and Varzi (see Kulvicki 2010a; Kitcher and Varzi 2000). 
This discussion will lead to the task of more seriously considering the way 
cognitive access is obtained to the content of visualisations, namely via per- 
ception. It has to be asked what kind of knowledge we acquire by making use 
of this epistemic source. What follows from the fact that we access pictorial 
information by vision as the primary sense of human beings to cognitively 
access the world? 

I will argue, and this is the final part of this investigation into the epi- 
stemic status of visual representations in science, that acknowledging the fact 
that the information process is based on perception in this way makes clear 
in what sense scientific images can actually be worth, in Kitcher and Varzi’s 
words, “2alephO” words (see Kitcher and Varzi 2000). Apparently there are at 
least two different approaches to spelling out these epistemic merits of visual 
representations. One will lead to a discussion of different kinds of knowledge 
and the question in what sense visual representations might be particularly 
helpful in their transmission. The other will be about scientific understanding 
and how visualisations can contribute to its achievement. 

The discussion will include a comparison of the three different represen- 
tational modes, namely the visual, the linguistic and the numerical. Whereas 
the second part strives for the rather moderate aim of showing that these 
different kinds of representation reveal comparable characteristics in the con- 
text of information transmission, the last part of the analysis is more ambi- 
tious. Here, the question will be pursued whether there is a kind of epistemic 
surplus of scientific images not realisable with the other vehicles of communi- 
cation under consideration. 


4.1 Visual arguments? 


As seen in the previous chapter, visual representations in the exploratory con- 
text might seem problematic for several reasons. Image manipulation, the 
artificial character of visualisations engendering a constructivist interpret- 
ation of their referents and the problem of theory-ladenness of observation 
have already been discussed. All of these difficulties dwell on the evidential 
status of visual representations in research processes. Thus, the question was 
whether scientists can be epistemically justified when referring to an image 
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of their object of research to justify knowledge claims about it. The analysis 
above made clear that no general worry about (digitally) manipulated images, 
a missing referential relation between visual representation and its object of 
depiction, or a theoretically induced misinterpretation of visual data can 
be maintained that would turn visualisations in science into a particularly 
unreliable source of information. On the contrary, a variety of reasons were 
identified showing that the scientific epistemic practices are reasonable, albeit 
fallible. 

But even though this result speaks in favour of the epistemic capacities 
of images in science, philosophical problems remain. Interestingly, these are 
expressed in particular with respect to the explanatory context, where visual 
representations are intentionally used to convey information. Although this 
potential of transmitting information motivates scientists to use images in the 
exploratory context too, philosophers tend to question this capacity of visual 
representation. In particular, they are sceptical of the ability of images to 
fulfil this epistemic task in argumentative contexts. 

Before going into the details about the exact nature of the philosophic 
contention here, let me briefly draw attention to an imbalance of the philo- 
sophical discussion. The point is that, although the mode of epistemic access, 
namely perception, to information encoded in visual representations remains 
the same in the explanatory and the exploratory contexts, its philosophical 
evaluation varies. Few, if any, philosophers would deny that perception can 
serve as an epistemic source. They do not contest the claim that accessing 
the information encoded in visual representations will perceptually allow the 
recipient to acquire at least some knowledge about what was supposed to have 
been transmitted by their means. However, the argumentative context refer- 
ring to this way of accessing information does not apparently suffice to make 
knowledge claims plausible which are based on this source of information. 
Thus, whereas perception seems to be an admissible source when it comes to 
scientific observations and experiments, the reasonableness of referring to it 
in the context of scientific arguments is questioned. 

Now, it might be argued that what is at issue is that in many instances there 
is a necessity to interpret scientific visualisations to understand their content 
correctly. However, two points can be mentioned as a rejoinder. 

Firstly, interpretations of what has been perceived also seem to play a role 
with regard to ordinary instances of perceiving the world. The more precise 
the observer’s background knowledge is, the more she will be able to discern, 
as Alan F. Chalmers makes clear when comparing a layman’s observations 
with those of a professional botanist travelling through the Australian bush 
(see Chalmers 1999, 11f.). He points out that, undoubtedly, the botanist will 
perceive more and also more detailed facts about the native flora than the 
layman will be able to notice. What explains this is that the botanist “has a 
more elaborate conceptual scheme to exploit” (ibid.) than the layperson. That 
is, the botanist has more background knowledge at her disposal than the lay- 
person with which to reason about what exactly she has observed. Thus, the 
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necessity of interpretation in the process of understanding is by no means a 
unique feature of visual representations. 

Secondly, background knowledge about the reliability of the instru- 
ment and about the mapping function that defines the informativeness of 
the resulting image with respect to the object depicted limits the spectrum 
of possible interpretations of scientific visualisations reasonably. Scientists’ 
reasoning is not driven by guesswork, but by these theoretical and practical 
restrictions. Hence, highlighting the relevance of interpretations to under- 
stand scientific images does not constitute a rationale framed to question their 
assumed epistemic capabilities. 

Actually, the bone of contention consists in something else. It is the 
question about the propositionality of the information transmitted that is 
critically discussed by philosophers, especially in the explanatory context. 
Epistemologists still tend to understand ‘knowledge’ primarily as ‘knowing- 
that’, namely propositional knowledge (see Grundmann 2008, 71).4 From this 
point of view, the question about the epistemic status of visual representations 
is essentially about their capacity to transmit propositional knowledge. 

In the following, philosophical worries concerning the epistemic status 
of visualisations in the explanatory context will be scrutinised more closely. 
Moreover, an alternative approach to the topic resulting from the previous 
considerations about how best to analyse visual representations in the explana- 
tory context will be discussed. All of these reflections will finally contribute to 
a better understanding of what kind of epistemic contributions visualisations 
can make with respect to scientific arguments. 


4.1.1 The philosophical challenge 


In the previous chapter, it has been argued that scientific images possess the cap- 
acity of information recording, storage and transmission. Contrary to scientists 
making use of these capacities in their epistemic processes, philosophers still 
call them into question in the explanatory context. What exactly are the diffi- 
culties that cause this sceptical attitude in philosophy? 

Apparently not all branches of philosophy are affected alike by these 
misgivings. Yet, as pointed out above, epistemologists and philosophers of 
science have shown crucial shortcomings in dealing with visual representations 
until rather recently. Two aspects are relevant to explain these difficulties in 
dealing with visual representations. On the one hand, there is a certain philo- 
sophical tradition which can be traced back to Plato and René Descartes, 
whose proponents question the suitability of images as epistemic means. On 
the other hand, a particular focus on (natural and formal) language in ana- 
lytic philosophy and, consequently, in the philosophy of science being based 
on this tradition, has to be mentioned in this context. 

The first aspect has often been pointed out as a difficulty in picture 
theory. I will therefore only briefly summarise the related line of reasoning. 
Perception is commonly regarded as a source of knowledge nowadays, but 
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this has not always been the case. In particular, so-called rationalists were 
rather sceptical about the reliability of sense perception as an epistemic 
source. It is here that Plato’s and Descartes’s misgivings can be subsumed. 
Both are convinced that observation is not able to yield knowledge and that 
we have to rely on reason instead if we are looking for a secure basis for our 
knowledge claims. Consequently, if perception is not admissible as a source 
of knowledge, perceiving images cannot fare much better from an epis- 
temological point of view. Plato explicitly discusses the case of images in 
his Politeia. Here he points out that pictures only mimic the entities of the 
real world, which themselves are only imitations of ideas. Consequently, 
images only show the appearances of appearances, but not the essence of 
things. This is why they are not suitable means to transmit knowledge (see 
Platon 2006, Politeia 597a-598d). 

Descartes is similarly sceptical about our perceptual abilities to acquire 
knowledge of the world. He discusses the unreliability of sense perception 
in his attempt to point out a secure foundation for our knowledge system 
(see Descartes 1994). He makes his worries particularly clear when analysing, 
as an example, what the essence of ‘wax’ might be (see ibid., 23f., Second 
Meditation). Here he demonstrates that we cannot rely on our perceptual 
abilities to answer this question, as apparently none of the characteristics 
that can be observed, tasted or felt remain unaltered if the wax is heated. 
Descartes infers from this that none of these properties belong to its essence. 
To put it differently, perception is not a reliable source of knowledge if we are 
interested in investigating those essential facts. Yet, if perception in general 
cannot be regarded as a reliable epistemic source, picture perception cannot 
be taken as an epistemic means either. Obviously, philosophers following this 
rationalist tradition will then be rather reluctant to take visual representations 
into account in their epistemological analyses. 

Jakob Steinbrenner addresses the second point mentioned above. He 
highlights the fact that most of all the focus on language of early analytic 
philosophers still influences philosophic discourse. He describes the resulting 
difficulties via a comparison of some of Gottlob Frege’s main theses about 
verbal expression with characteristics of images (see Steinbrenner 2009, 285f.). 
What becomes clear is that none of the main features of verbal expressions — 
for example, being truth-bearers or being objects of logical operations such 
as expressing inferences, entailments, negations — show (at first glance) an 
analogous counterpart in visual representations. Steinbrenner adds that, even 
though none of these theses has remained uncontested by contemporary ana- 
lytic picture theorists,° this heritage of the philosophical founding fathers, 
so to speak, has long hindered a thorough tackling of the topic in analytic 
philosophy (see ibid., 286). Yet what exactly did Frege say that caused these 
problems? 

In The Thought. A Logical Inquiry (Frege 1956),’ Frege deals with the topic 
of truth. In particular, he is interested in the question concerning what entities 
can be bearers of truth values. He suggests that it is only “thoughts” that can 
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fulfil this task. But what exactly is it that he calls a “thought”? Frege offers the 
following description: 


[w]ithout wishing to give a definition, I call a thought something for 
which the question of truth arises. [...] the thought is the sense of the 
sentence without wishing to say as well that the sense of every sentence is 
a thought. The thought, in itself immaterial, clothes itself in the material 
garment of a sentence and thereby becomes comprehensible to us. We say 
a sentence expresses a thought. 

(Frege 1956, 292) 


From his point of view, a thought is then an abstract object. 

It is already inherent to this short summary of Frege’s theses on thoughts 
what constitutes the basis for our problems with respect to scientific images. 
The two relevant phrases are ‘bearers of truth values’ and ‘sense of a sen- 
tence’. Why are these two characteristics of thoughts so problematic in the 
current context? 

Firstly, with regard to truth, Frege explicitly states that it is truth at which 
scientists aim. “What are called the humanities are more closely connected 
with poetry and are therefore less scientific than the exact sciences which are 
drier the more exact they are, for exact science is directed toward truth and only 
the truth” (ibid., 295, my italics). Consequently, if we are interested in scien- 
tific endeavour characterised in this way,* we have to focus our attention on 
those entities that can contribute to this attempt. This introduces the second 
point of concern derived from Frege’s account. 

The question is, what kinds of entities are possible candidates for our ana- 
lysis and, in particular, whether visual representations are amongst them. 
There are two lines of reasoning related to this question in Frege’s text. One 
is explicitly about images, the other about bearers of truth values. Frege 
questions whether our common habit of calling pictures true is justified (see 
ibid., 290ff.). He points out several aspects in this regard. In particular he 
makes clear that it is not a material object that we call true, but a represen- 
tational relation intentionally brought about (see ibid., 290). This is why we 
use the adjective ‘true’ regarding pictures but not regarding other material 
objects such as stones. To be concise, Frege suggests that our practice of 
calling pictures true is a comprehensible, albeit illegitimate, extension of the 
scope of the term ‘true’. He states that “what is improperly called the truth of 
pictures [...] is reduced to the truth of sentences” (ibid., 291f., my italics). It 
is not pictures that can be called true, but sentences — or, more precisely, the 
sense of sentences, as Frege claims. 

Gombrich reiterates this point in picture theory, when he states that in 
the same sense that verbal assertions cannot be called red or green, pictures 
cannot be called true or false (see Gombrich 2004, 59).? It seems as if visual 
representations belong to the wrong category and thus cannot be considered 
seriously when epistemologically reflecting on the pursuit of truth in science. 
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From Frege’s point of view, they cannot be the bearers of truth values, and 
thus cannot contribute to the proclaimed epistemic aim of science. 

However, this critical stance towards visual representations in epistemic 
contexts derived from Frege’s account did not remain uncontested. As there 
will be a discussion in detail concerning the possibility of visual arguments, 
and thus the connected question about the truth-bearing capacities of sci- 
entific images, in the following two sections, I will only briefly mention, as a 
concluding remark, two philosophers who argue for a positive status of sci- 
entific images. 

Firstly, there is Marcia Eaton, who explicitly tackles the question about 
truth values of visual representations (see Eaton 1980). Similar to my 
approach, she suggests regarding images as parts of communicative acts. She 
also makes use of Kjorup’s suggestions (see ibid., 16), mentioned in our ana- 
lysis above. It is by this embedding of visual representations in communica- 
tion that they can be regarded as (parts of) assertions. Now, Eaton argues 
that it is these assertoric acts that are evaluated along the lines of truth and 
falsehood. In detail, her approach amounts to the following thesis: 


[t]hus pictures lie somewhere between interpreting a set of symbols 
and interpreting a state of affairs. Judging that a picture is true or false 
necessitates the viewer’s taking an active role in which he or she first 
formulates a statement which he or she believes to be a possible interpret- 
ation of the picture and then relates that statement to his or her beliefs. 
(Eaton 1980, 21) 


Here, Eaton states her conviction that the content of images has to be 
translated into linguistic expressions before the question of truth can be 
tackled. In this sense, she tries to evade the difficulties pointed out above in 
a twofold sense, namely (a) by claiming that images can be translated into 
sentences and (b) that this translation, being based on the embedding of 
images in communicative acts, has the recipient judge what she believes the 
producer of the image intended to express with its aid. 

Eaton’s strategy seems to fit nicely with the characteristics of the explana- 
tory context of science. Here, recipients indeed try to figure out what the 
intended message of a communicative act might have been. Yet, as we will see 
in section 4.1.3, this is only part of the meaning that can and will be extracted. 
Moreover, Eaton’s approach is somehow at odds with what happens in the 
exploratory context of science. Investigating the photograph of Olympus 
Mons (see Figure 2.12), the scientist will not be interested in the intention of 
the photographer, but what this image can tell her about the height profile of 
this volcano on Mars. 

The second scholar to mention is David C. Gooding (see Gooding 2010). 
He is not so much concerned with questions about truth as with processes 
of visual reasoning in science. Nonetheless, he points out that hypotheses 
based on visual representations can be empirically validated by “checking for 
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correspondence between features derived from models and observed features” 
(ibid., 28). Assuming that a correspondence theory of truth rules here does 
not seem far-fetched. Without addressing this issue in any critical way, 
Gooding spells out how scientists make use of visualisations to derive hypoth- 
eses needed for explanations and predictions in their domain of research. In 
particular, he points out that scientists constantly manipulate their represen- 
tational means. That is, he argues that it is not a particular image that is rele- 
vant to the scientist’s reasoning, but a process involving different images (see 
ibid., fig. 10 on p. 27). 

What Gooding explains here suggests an analogy to Vertesi’s descriptions 
of how scientists work with rover images taken on Mars (see Vertesi 2015). She 
points out how working with those photographs, namely by digitally manipu- 
lating them, for example highlighting structures by colouring them or adding 
numerical or verbal explanations, converts them into maps of Mars (see, for 
instance, ibid. fig. 4.1. on p. 109), thereby transforming those images into tools 
not only to manoeuvre the rover geographically on the surface of Mars, but 
also in accordance with the tasks it is supposed to perform, for example, to 
analyse certain samples of soil (see ibid., 116ff.). Gooding describes a similar 
process at a more abstract level. He claims that: 


[m]oving from interpreted sources to structural models and on to pro- 
cess models generates visual theories that satisfy the explanatory 
aims of science. This transformation from simpler to more complex 
representations increases information content, enabling models to incorp- 
orate more domain knowledge. 

(Gooding 2010, 25) 


Obviously, then, what allows scientists to draw inferences on the basis of 
visual representations is their embedding into larger processes also containing 
other kinds of representation. 

In a similar way that Kjorup’s suggestion draws attention to the wider 
context of image usage, Gooding’s account reasonably broadens the focus of 
attention by calling to mind that scientific investigations and problem solving 
is a constant process rather than a single judgement. Helpful as this insight 
might be, there are nonetheless two critical remarks to be made about his 
proposal. On the one hand, his approach still leaves us with the question of 
how exactly, contrary to Frege’s suggestion, images can be said to transmit 
information and thus constitute the basis of those processes of reasoning 
described by Gooding. On the other, his talk about ‘models’ in this context 
brings in another confusing issue, because it suggests the question of whether 
images can and should be understood as models, whereby the term ‘model’ 
itself is highly contested in the philosophy of science (see e.g. Bailer-Jones 
2009; Hesse 1970; Morgan and Morrison 1999). 

From this point of view, it seems advisable to set our focus back on scien- 
tific images and questions concerning their epistemic capacities. This is then 
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the point at which to elaborate on Laura Perini’s work on the topic. Contrary 
to Eaton, she defends the thesis that no prior translation of images into lin- 
guistic expressions is necessary to judge them along the lines of truth and 
falsehood. 


4.1.2 Laura Perini on visual representations in scientific arguments 


Not least because of Laura Perini’s important contributions to the debate 
(see Perini 2005a; 2005b; 2005c; 2010; 2012a; 2012b; 2012c), philosophers 
of science have finally started discussing what the ubiquity of visual 
representations in science means in epistemological terms. Acknowledging 
this inspiring role that Perini’s work has played for the philosophical discus- 
sion — and likewise for this book — her suggestions concerning the roles and 
status of visualisations in epistemic processes of science will be examined 
more closely now. 

With respect to visual representations in the explanatory context, 
she states that philosophers tend to regard them as “‘mere illustrations’ 
that are redundant expressions of information presented in the text, or 
convey information inessential to the argument” (Perini 2005c, 913). Yet 
it seems quite unlikely that scientists would put such an emphasis on 
visual representations if they were that useless in the epistemic context of 
argumentation. 

Perini not only makes us aware of this tension in the assessment of image- 
practices by scientists and by philosophers, she also tries to offer a different 
evaluation of the epistemic capacities of scientific images. Here, Marianne Ina 
Richter, who critically discusses Perini’s attempts in her PhD thesis,'° makes 
clear that there are apparently three different strategies that Perini could 
have chosen to make her point. She could have defended the view (i) that 
images are arguments, or (ii) that they are functional parts of arguments, or 
(iii) that visual representations contribute to arguments without being parts 
of them (see Richter 2014, 159). I agree with Richter that Perini selects the 
second option (see ibid., 165ff.). Yet it has to be added that Perini presents 
two different lines of reasoning in this respect. Firstly, she tries to show that 
images can indeed be proper parts of scientific arguments. Secondly, she also 
analyses whether there are particular epistemic tasks that can be fulfilled 
exclusively by visual representations. 

Perini’s first line of reasoning is deeply intertwined with the question 
about the truth-bearing capacity of images. She states the difficulty con- 
cisely: “[p]hilosophers define arguments in terms of sets of statements. This 
may explain why philosophers of science have paid so little attention to fig- 
ures” (Perini 2005b, 262f.). Apparently images simply belong to the wrong 
representational category to play the role of premises or conclusions or, at 
least, proper parts of such.'! Moreover, Perini explains why the difference 
between visual and linguistic representations is commonly regarded as so 
important in the context of argumentation. 
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The support that premises provide a conclusion is analyzed in terms of 
validity or strength, and soundness, so any representation that is an inte- 
gral part of an argument must be one to which those features could be 
relevant. Validity, strength, and soundness are understood in terms of 
the truth conditions of premises and conclusions, so representations that 
contribute to arguments must have the capacity to bear truth. To show 
that figures are nontrivial parts of scientific arguments, the first question 
that must be addressed is whether visual representations can be true or 
false. 

(Perini 2005b, 263, my italics) 


Assessing arguments in a logical way presupposes that we are able to say 
something about (a) whether what is expressed in their premises and the con- 
clusion is true or false, and (b) whether what is derived in the conclusion is 
already entailed in the argument’s premises. However, these are exactly the 
logical characteristics that, following Frege, visual representations apparently 
do not possess. 

Now, Perini is convinced that this last commonly held assumption is false. 
She tries to show that visualisations actually can be truth-bearers (see Perini 
2005b; 2012c) and thus can be regarded as (proper parts of) premises or 
conclusions of scientific arguments. Her line of reasoning works as follows. 

Firstly, she discusses some examples from logic and mathematics, in par- 
ticular from set-theory and geometry (see Perini 2005b, sect. 1). She points 
out that, in this realm, there are some visual representations that can indeed 
bear truth values. She refers to Venn, Euler and Peirce diagrams and states 
that they allow “truth-preserving inferences from diagram to diagram” 
(ibid., 264).'? To illustrate this point, take the classical example of the syl- 
logism which entails a universal statement as one of its premises. As has 
been pointed out in section 2.1.4 of this book, such a universal statement 
can be expressed by Venn diagrams. Figure 4.1 shows how the universal 
statement ‘All men are mortal beings’ can be expressed by using a diagram 
instead of a sentence and, thus, how a diagram can work as a premise of 
the syllogism. 

Furthermore, Perini points to the existence of visual proofs of certain math- 
ematical questions, in particular in Euclidian geometry (see ibid., 265f.).13 In 
what sense images can play a role in mathematical explanations is shown by 
Max J. Kobert, who demonstrates how visual representations can be used to 
explain algebraic statements (see Kobert 2010, 135f.). For instance, the for- 
mula (a+b)? = a? + 2ab + b? can be explained by considering Figure 4.2. That 
is, the algebraic formula is explained by turning it into a geometrical problem 
which can be solved visually. 

Yet despite these initially positive results, Perini sets them aside by claiming 
that “[b]ecause scientific arguments are significantly different from the deductive 
diagrammatic systems [...], we cannot extrapolate from their results to the 
figures that appear in contemporary research journals” (Perini 2005b, 267). 
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Socrates is aman 


Socrates is mortal 


Figure 4.1 Syllogism containing Venn diagram, general statement (All men are mortal 
beings) expressed by Venn diagram. 


Source: own image. 


From my point of view, she dismisses too much here because, although Perini 
is right in claiming that not all scientific visualisations are of the kinds used in 
logics and mathematics, some are. 

With respect to images used in the arguments of the empirical sciences, 
some points have to be made clear in advance. (1) Perini thinks of depic- 
tion along the lines of Goodman’s theory as explained in section 2.2.2 of 
this book. That is, images are parts of symbol systems whose difference to 
linguistic symbol systems is characterised on the syntactic level (see ibid., 
267ff.). (2) She rejects resemblance relations as the relevant feature to discern 
what is meant by ‘depiction’ (see ibid., 268). (3) Yet she admits that similarity 
might still play a role. She points out that, although it is by conventions 
that the characters of a given symbol system are ascribed their meaning, 
it can be the case that some conventions refer to resemblance relations to 
determine the meaning. “Content can also be determined by conventions 
that relate symbol and reference through resemblance relations” (ibid., 268). 


a + b 
{| 


Figure 4.2 Pictorial explanation of algebraic formula ((a+b)* = a? + 2ab + b’). 
Source: own image. 
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(4) These conventions belong to particular symbol systems. That is, if an indi- 
vidual image is interpreted as depicting y because it resembles y, we do so 
because this image belongs to a particular symbol system that entails resem- 
blance relations as interpretative rules. (5) It is not by translating images into 
sentences that they are assigned a meaning. (6) It is not the meaningful content 
of a sentence that is assessed as being true or false when images are considered 
(see ibid., 274). (7) It is a feature of the respective system, not of individual 
representations, that truth values can be ascribed to the latter (see ibid.). 

Taking these points as the background to her considerations, Perini then 
makes use of Alfred Tarski’s theory of truth to defend her claim that images 
can bear truth values. Now, the classical Tarskian schema of truth is: “x is 
true in L iff p”. The standard example is: “ ‘snow is white’ is true iff snow is 
white”. Thus, with respect to visual representations, we now have to make 
clear, what ‘x’, ‘L’ and ‘p’ refer to. It is here that, despite her initial dismissal 
of linguistic expressions as mediating representations, Perini claims that we 
are in need of names and sentences to make statements about 


the definition of truth for a visual system. 
(Perini 2005b, 276) 


She points out that such a definition “requires a way to assign a linguistic 
name to each symbol, based on its structural form, and also a linguistic expres- 
sion of the content of each representation. A definition of truth for a visual 
symbol system consists of a statement of this form for every figure f in a 
system: Name(f) is true IFF statement(f)” (ibid., 276). Thus, to make claims 
about the truth-bearing relation between ‘x’ and ‘p’, that is, between a symbol 
(for example an image) and its meaning, both have to be expressed linguistic- 
ally. Moreover, it is by the interpretive rules of the system to which the symbol 
belongs that the meaning of the respective symbol is fixed. Thus, Figure 4.1 is 
true, if and only if, all human beings (premise 1) are mortal (premise 2). The 
conventional rule that helps us to interpret this diagram consists in the following 
statement: interpret the sets of two intersecting circles (1, 2) as identical if all 
elements of 2 belonging to the set of 1, that is, are entailed in the intersecting part 
of l and 2, and no element of 2 is left outside this intersecting part so that the 
remaining part of 2 is shaded, thus marking an empty set. Interpretative rules 
could also be defined at an even more basic level here when the shaded part is 
stated to mean an empty set or the intersecting part of both circles a shared set. 

Interpretative rules such as those last statements then connect an image or 
a symbol (x), named e.g. by calling it ‘Figure 4.1’ or ‘intersecting part’, with 
its meaning (p) within a particular symbol system (L). ‘Meaning’ is thereby 
understood as the reference of the symbol which can be an object, but also 
a state of affairs as in the example above. Interpretative rules then determine 
the truth conditions of a particular diagrammatic symbol. It is in this sense 
that Perini thinks that Tarski’s theory is a suitable means by which to discuss 
the truth values of images. 
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Tarski’s work shows that even though there is no general theory of truth 
with which we can test individual visual representations for the capacity 
to have truth value [...], certain symbol systems are characterized by a 
systematic relation between symbol form and referent, in which the truth- 
conditions of symbols are a function of their form. 

(Perini 2005b, 275) 


Admittedly, such an account works very well with respect to highly 
conventionalised symbol systems such as the diagrammatic. However, as 
Perini points out herself, many images in science are not of this kind (see ibid., 
279). As an example, she discusses electron micrographs (see ibid.). Despite 
the difficulties that she highlights with respect to images of this kind, she is 
optimistic that it is still possible to define a concept of truth also holding in 
pictorial symbol systems. 

What x (symbol), p (meaning) and their relating rule are in this context 
has to be re-analysed in order to discover the cause of the problems here. The 
mapping of symbol and meaning becomes much more difficult if images are 
considered such as the micrograph, that is, images of a less conventional style 
in comparison to diagrams. In particular, there are many characteristics that 
are not related to any linguistic expression. Consequently, truth conditions 
mapping symbols to states of affairs cannot be formulated in the way 
suggested with regard to the diagrams above. 

Therefore, Perini suggests that it is relevant to consider “the form of the 
image as a whole” (ibid., 280). That is, the micrograph represents the scanned 
sample as being of a particular shape — this is what Perini calls the “form of 
the image” and which constitutes the x of the Tarskian schema. The meaning 
(p) is the actual shape of the sample. The truth condition of an electron micro- 
graph reads as follows, “an electron micrograph is true IFF the shape of the 
micrograph is a geometric projection of the shape of the sample scanned in 
producing the micrograph” (ibid., 280). 

Yet despite this explanation of how to deal with images of this latter 
kind, one might object that her strategy presupposes the translatability of 
pictorial symbols into linguistic expressions to check for the relevant rela- 
tion. In particular, this problem comes to mind when she contemplates the 
difficulties of naming entities (see ibid., 282). Many visual representations 
in science display characteristics, such as those depicted by the micrograph, 
that cannot be translated into linguistic expressions because the latter are 
simply not available. There are no names, not even linguistic descriptions, at 
the scientists’ disposal either to name the symbol or to describe its meaning. 
If the content of such visual representations cannot then be translated and 
thus stated by making use of linguistic expressions, does this imply that 
the method employed to define truth conditions explained above is not 
applicable here? 

Perini is aware of this objection. She claims that there are two reasons 
speaking against this negative assumption: 
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[flirst of all, this investigation was launched to show whether non- 
linguistic representations can bear truth or not. This question is begged 
by invoking the assumption that only representations whose content can 
be expressed with a linguistic form of representation have the capacity 
to bear truth. So, the question of whether a micrograph could be true or 
false cannot be settled in this way. Second, that stance does not fit with 
the fundamental intuition that truth value depends on reference to states 
of affairs, since micrographs not only represent states of affairs, but do 
so without their content being assigned to them through the mediation of 
another symbol system. 

(Perini 2005b, 282f.) 


The first rejoinder seems to me to be a rather weak one. One can state 
vice versa that the necessity of translation indeed shows that pictorial 
representations cannot bear truth values. The consequence then would be 
that the project of defining a concept of truth valid for pictorial symbol 
systems has failed. 

Perini’s second rejoinder, however, is more substantial. Pointing out that 
truth values are connected to states of affairs and that micrographs represent 
such states of affairs apparently suggests itself as a criterion of evaluation. 
How does the representation of states of affairs work, then, in the pictorial 
realm? She argues that images represent states of affairs because what is 
represented by them is related to the latter by conventional rules that guide 
the interpretation of those images. This also explains why she thinks that 
images can represent not only visible features of their object of reference, but 
also invisible ones. She states that: 


[v]isual representations are defined by the fact that spatial features of the 
symbol are interpreted to refer to features of the referent — they are not 
defined in terms of similarity with their referents. For this reason, visual 
representation is not restricted to visible subject matter. 

(Perini 2010, 137) 


William Goodwin takes a different track in criticising Perini’s account of 
the truth-bearing capacity of visual representations (see Goodwin 2009). He 
is also sceptical about the theory of depiction and the theory of truth chosen 
by Perini. However, his main critique is focused on “the idea that the only way 
to understand visual representations as having a ‘genuine’ role in scientific 
discourse is by finding some way to understand these representations as cap- 
able of being true or false” (ibid., 374). Goodwin shares Perini’s conviction 
that visualisations can play essential epistemic roles in scientific discourse. 
However, contrary to her approach, he thinks that this important task can 
be explained without struggling with their presumed capacity to bear truth 
values. How then does Goodwin conceive of the precise epistemic function of 
visual representations in scientific arguments? 
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The starting point of his objection is his asking what the expression 
‘contributes to an argument’ might mean. From Perini’s point of view, making 
such contributions implies that the respective entity plays the role of (parts of) 
premises or conclusions. As Goodwin claims, such a conception means that 
“a representation ‘contributes to an argument’ only by being in logical or evi- 
dential relationships with the other representations in the argument” (ibid., 
376). Postulating such relationships then motivates the search for adequate 
conceptions of truth in this context, since the other relevant representations 
are linguistic expressions. However, as Goodwin points out, there is also 
a second sense in which the statement ‘contributes to an argument’ can be 
understood, namely as “simply being important to the expression of the argu- 
ment” (ibid.).!4 

It is this second sense that Goodwin highlights as relevant when considering 
particular kinds of representations, for example, the functions of words 
or numerical data in the context of scientific arguments. This second sense 
of ‘contributes to an argument’, however, does not require that the entities 
analysed in this regard have to be capable of bearing truth values. Thus, 
Goodwin infers that: 


Perini has provided no general philosophical reason to suppose that 
the visual representations used in science must be understood to bear 
truth. Recognizing that such representations contribute to scientific 
discourses in which they occur leaves open the question of how they 
contribute. 

(Goodwin 2009, 377) 


Moreover, Goodwin highlights the fact that, even though scientific images 
might in some instances play the epistemic role that Perini ascribes to them, this 
does not preclude the possibility that their roles are different on other occasions. 

To support his view, Goodwin discusses the example of structural formulas 
in organic chemistry. He suggests that they can be regarded as diagrams, and 
thus as visual representations in Perini’s sense. Now, with respect to their 
functional roles in scientific discourse, Goodwin points out two different tasks 
that they fulfil. 

On the one hand, he claims that they are used as descriptive names in sci- 
entific papers (see ibid., 378). This he derives from their interchangeable use 
with IUPAC labels!‘ (see ibid., 380), that is, labels that are generated by using 
the rules provided by the nomenclature developed by this union. 

On the other hand, Goodwin argues that structural formulas also serve the 
function of models in scientific texts. He writes: 


[o]f course, it is not possible to reproduce physical models of organic 
compounds on the printed page, and so structural formulas stand in 
for physical models in chemical discourse. [...] Structural formulas — 
supplemented with some additional conventions — are, on the other 
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hand, objects whose characteristics can be used to infer properties of the 
compounds they denote. 
(Goodwin 2009, 381) 


Thus, Goodwin thinks that structural formulas can be manipulated to infer 
information about the chemical compounds that they represent. 

Regarded this way, structural formulas as visual representations provide 
essential information, but it would be wrong to ascribe truth values to them. 


When used as a model it is facts that are true of, or in, the structural for- 
mula that make it so useful in the discourse of organic chemistry. So when 
a structural formula is used as a model, it is not true or false. Rather, 
certain claims are true or false in virtue of the structural formula, and 
these claims license conclusions, perhaps by way of assumed similarity 
relations that obtain between the relevant objects, about the chemical 
compounds that it denotes. The role of structural formulas in this context 
is therefore that of truth maker, not that of truth bearer. 

(Goodwin 2009, 388) 


From Goodwin’s point of view, his example shows that Perini is wrong 
in assuming that visual representations can make epistemically relevant 
contributions to scientific discourse only if they are capable of bearing truth 
values. Contrary to this, he argues that at least some visualisations function 
as models and, in this sense, make certain claims put forward in this discourse 
true or false. 

Now, there are several questions to be asked about Goodwin’s account. 
Two aspects, also discussed by Perini in her rejoinder to Goodwin’s critique 
(see Perini 2012c), concern, firstly, the scope of his claim and, secondly, the 
question of whether his example shows what it is supposed to demonstrate. 

I will start by discussing Perini’s second rejoinder first. She calls into 
question whether Goodwin’s example, namely whether those structural for- 
mulas published in papers in organic chemistry, really function as models. 
She emphasises Goodwin’s thesis that it is the possibility of manipulating the 
parts of the model to gain additional knowledge by using it (see ibid., 144). 
However, as a 2D representation in a scientific text, those structural formulas 
cannot be manipulated in the sense intended by Goodwin. “All the diagrams 
Goodwin discusses are static markings on a flat surface. The positions of 
their parts can’t be manipulated” (ibid., 145). In this sense, the analogy that 
Goodwin tries to establish with regard to models and scientific visualisations 
does not hold from Perini’s point of view. 

Nonetheless, she does acknowledge a certain similarity with regard to their 
epistemic function, but only via a detour through imagination. She states: 


[t]he viewer can imagine a three-dimensional object after viewing the dia- 
gram, and then imagine what that three-dimensional object would look 
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like if its parts were rotated. Then they could use the same reasoning that 
was applied to their observations of the actual three-dimensional models 
after they were manipulated. [...] If this is how the diagrams are used, 
then some model-based reasoning is involved, but the diagram itself is 
not functioning in the same way as the model in that reasoning process; 
instead, the imagined compound functions like the model. 

(Perini 2012c, 145) 


Now, Perini uses this description of how the analogy between inferences 
made with the aid of models and by using scientific visualisations can be 
preserved to support two different claims of hers. (1) She points out that, 
with regard to the particular example discussed by Goodwin, the analogy 
does not hold. That is, he has not shown that, in this particular instance, 
scientists were interested in gaining new knowledge via mentally manipu- 
lating the presented structural formula. On the contrary, she thinks that the 
diagrams in question actually work in the sense she suggested earlier, namely 
“to make a claim about the structures of the isomers” (ibid.). (2) Perini admits 
that visualisations do have the capacity to function in the way intended by 
Goodwin and that some of them are also used in this way, namely as tools in 
visual thinking (see ibid.). 

Perini’s rejoinder seems convincing. Obviously, visual representations can 
serve the function of tools of thinking, as has been pointed out in section 
3.1.1 of this book. However, it can be questioned (and Perini does so) whether 
it is this functional role that is intended when distributing the respective visu- 
alisation in a scientific paper. More often than not, the presentation of results 
seems to be relevant, namely the evidential role that Goodwin also mentions 
in his text. This topic will be discussed in due course. 

Before going into those details, the analysis should be concluded by also 
examining Perini’s first rejoinder to Goodwin’s critique more closely. Although 
I agree with her first reply, I do have some worries concerning the scope of 
her and also of Goodwin’s claim. Both emphasise the functional roles of the 
scientific images they discuss in their respective articles as the more or less 
predominant ones. This also seems to be the rationale why Perini claims, for 
example, that “[...] the fact that in some circumstances scientists use images in 
ways that are best described as using the image as an object (or description- 
fitter), doesn’t imply that most scientific images are used this way” (ibid., 146, 
my italics). What is missing in both accounts is a more precise analysis of 
contexts of usage as presented in section 3.1 of this book. Such a more detailed 
perspective not only reveals that an epistemology of scientific visualisations 
can accommodate both accounts of functional roles without devaluing the 
relevance of either of them in the scientific context, but also that, even if 
the discussion is focused on the explanatory context alone, as both Perini’s 
and Goodwin’s are, it has to be acknowledged that visual representations can 
fulfil different epistemic and non-epistemic functions. It depends on the aim 
pursued and the mode of the communicative act as well as on the audience 
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involved in it that determine the exact nature of their function. I reject, there- 
fore, the predominance of the respectively identified functional roles of visual 
representations expressed by Perini and by Goodwin. 

Furthermore, it might be helpful to mediate between Perini’s and 
Goodwin’s account to bring to mind that there is another close connection 
between models and visualisations that is often overlooked. Végtli and Ernst 
remind us of there being many instances of physical models reproduced in 
visual representations. The fact that, in the processes of copying such images, 
people pay less attention to what has been the actual object of reference of 
their template motivates Végtli and Ernst to urge their readers to be careful in 
interpreting the content of visualisations as showing real entities (see Végtli 
and Ernst 2007, 47ff.). 

To return now to the point that visual representations play an evidential 
role in scientific discourse. Goodwin puts forward an interesting idea en pas- 
sant. In a footnote of his paper he writes: 


[...] consider a piece of physical evidence in a legal trial. Surely it doesn’t 
follow from the fact that this physical evidence plays an important role in 
the prosecutor’s argument that the physical evidence itself must be cap- 
able of bearing truth. 

(Goodwin 2009, 375, ftn. 2) 


He mentions this example to support his initial thesis that certain entities 
can contribute to arguments, although they are not capable of bearing truth 
values. From his point of view, such physical pieces of evidence should be 
regarded as objects that make statements about them true or false. This is the 
parallel that he wants his readers to draw between this example and his claims 
about the role of visual representations in the context of scientific arguments. 

Although Goodwin’s suggestion offers an interesting new perspective on 
how to conceive of the role of visual representations in science, it still implies 
that, if scientists want to make use of images in argumentation, they have 
to formulate statements about them and use the latter in their texts. They 
still need verbal statements that visual representations can render true or 
not in order to express their arguments correctly. Yet, contrary to Goodwin, 
although in accordance with Perini’s point of view, the scientific practice of 
publishing visualisations as parts of scientific papers suggests that scientists 
do not see the need to translate those visualisations so that they play the roles 
they are supposed to fulfil. Information given in images is not recorded in the 
text of the article containing them. Usually there are only references in the 
text telling the reader to consider the relevant figure in order to get the infor- 
mation relevant to understanding the explanation offered. 

This seems to be particularly true with respect to genuine research papers in 
science. Articles addressing laypeople commonly contain descriptive sections 
about their visual representations to guide their novice readers in interpreting 
them correctly. Although scientific images are supposed to play an evidential 
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role in both contexts, their embedding within the written text of the respective 
article will be different. 

Perini is clearly vindicated in pointing out that scientific practices show 
that visual representations can play an epistemic role in their own right, that 
is, without prior translation. However, as I am also convinced that Goodman’s 
account does not offer the means suitable to explain the nature of scientific 
visualisations, I cannot follow Perini’s suggestion to regard the latter as truth- 
bearers as stated in her theoretical framework. Nonetheless, there are alter- 
native ways that might guide our analysis to conclusions similar to those 
intended by Perini. 

Two such strategies will be considered in more detail in the following 
sections. One is concerned with the cognitive content of visual representations. 
If it can be shown that visualisations do contain such a content, it might be 
possible to argue that this content works in a way similar to Fregean ‘thoughts’ 
when transmitting information. Hence, a subsequent analysis has to focus on 
the question of whether this kind of content is propositionally structured. 
If it turns out that it is indeed propositionally structured, a truth-orientated 
approach, such as the one preferred by Perini, can also be established via 
this route. 

The second strategy, although related to the first one in certain respects, 
follows a different line of reasoning. Close attention will be paid to the role 
of visual perception when cognitively accessing the information encoded in 
scientific visualisations. In this context, the epistemic qualities of perception 
and image perception in particular will be examined. Is knowledge gained via 
perception and, if so, what kind of knowledge? Is it propositional or non- 
propositional in kind? 

Perini suggests a strategy in a different paper (see Perini 2005c), similar to 
the idea just sketched, to account for the evidential role of scientific images 
in the explanatory context. Since she also refers to perceiving those visual 
representations as a relevant step in the process of their cognitive processing, 
it seems reasonable to combine the analysis of her thesis with a brief review 
of the discussion of perception as an epistemic source. 

Yet before going into the details of the two suggested strategies, this section 
about “visual arguments” should be concluded. The level of argumentation as 
a communicative act will be considered next. Moreover, analysing the topic 
in a broader perspective such as this is also a consequence of Kjorup’s thesis 
that the object of investigation should be whole communicative acts and not 
visual representations in isolation if their meaning is to be correctly under- 
stood. Taking this suggestion into account, how should visual arguments be 
conceived? 


4.1.3 Giving reasons, drawing conclusions 


In this section, a couple of ideas will be used to explain how visual representations 
can contribute to scientific arguments. My perspective on this topic is guided by 
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insights from semiotics, the philosophy of language and argumentation theory. 
What analytical tools do these approaches provide to analyse the role of images 
in this context? To begin with, they permit a better understanding of what the 
terms ‘argument’ and ‘argumentation’ mean; in particular, what parts they are 
composed of. Additionally, they will offer some useful information about the 
questions to which of these parts visual representations can contribute and how. 

We will start (1) with a brief analysis of Scholz’s theory of image games. 
This theory helps us to explain the effects of argumentation on particular 
images in science. I will call one of these effects a ‘layering of meaning’; that 
is, subsequent uses of one particular image in different contexts can add new 
dimensions of meaningfulness to the visual representation at hand. Despite 
this layering, Scholz’s theory also allows the conclusion that a kind of basic 
meaning will still be maintained in the encoded content. 

If itcan be conceded that such a layering of meanings can take place in argu- 
mentation, the question arises how best to analyse this process. This (2) will 
be examined next by deconstructing the act of argumentation including visual 
elements. As an auxiliary means employed in the investigation, Kjorup’s pro- 
posal of pictorial speech acts will be used. This analytic approach will provide 
the tools to describe the different components of argumentation as a commu- 
nicative act in science. 

Finally (3), this line of reasoning will stress the question of how to con- 
ceive of the propositional content inherent in the act of argumentation. To 
answer this question, I will discuss approaches from argumentation theory 
dealing with the topic of visual arguments. Two different ways to handle 
the subject will be considered: on the one hand, some scholars suggest that 
images do indeed have a propositional content that can account for the task 
of information transmission in question; on the other, there are philosophers 
who try to evade the question about propositionality or, respectively, who 
clearly deny that argumentation presupposes the propositionality of its parts 
by any means. 

Thus, the content-level of visual arguments will finally be returned to. 
Yet the line of reasoning leading to it points out the possibility of ana- 
lysing the contributions of visual representations when focusing on the level 
of argumentation — the communicative act — rather than on the content- 
level of individual images. To begin with, Scholz’s theory of image games 
demands closer attention. With this theoretical approach, Scholz suggests 
the term of image games in close connection with Ludwig Wittgenstein’s 
theses about language games. This latter approach is meant to explain how 
linguistic expressions acquire their meanings in conversational settings (see 
Scholz 2011). 

As Scholz adopts Wittgenstein’s conception for discussing how meaning is 
attached to images, it would be reasonable to first point out what he considers 
to be the essentials of language games. Scholz states that (1) the usage of signs 
is deeply intertwined with human actions. He points out (2) that words and 
sentences are not only embedded in contexts of sign usage, but also in cultural 
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contexts. Both contexts are characterised by certain rules that are relevant 
to the meaning of those linguistic expressions. The thesis is highlighted that 
(3) to understand linguistic expressions correctly it is necessary to know the 
rules governing the usage of signs in the different contexts of their usage 
and within the different cultural contexts.'* And finally (4) no claims about a 
presumed essence of language games are put forward by Wittgenstein who, on 
the contrary, defends the thesis that a variety of phenomena subsumed under 
this label is connected via family resemblance alone (see ibid., 369). These 
are then the four characteristics of language games on which Scholz relies 
to transfer Wittgenstein’s concept to the pictorial realm and that he uses as a 
stepping stone to develop his theory of “image games”. 

As was pointed out in section 2.2.2, Scholz is an adherent of Goodman’s 
theory of depiction. Consequently, he argues that the practice of using an 
entity as a picture implies that this entity fulfils Goodman’s definition (see 
ibid., 371) — that is, images belong to symbol systems that are characterised 
as syntactically dense and relatively replete. Nothing can be regarded as a 
picture per se. It is due to our treatment of a particular entity that causes it 
to be thought of subsequently as a picture. However, not any entity can be 
called an image in that way. In addition to being embedded within the relevant 
practices, it has to fulfil further conditions, namely the conditions of depic- 
tion mentioned by Goodman. 

It might be objected that this concept of the term ‘image’ is somehow in 
conflict with the anti-essentialist thesis that Scholz wants to transfer from 
Wittgenstein’s account. If all the entities to be called ‘images’ have to fulfil 
Goodman’s conditions of depiction, their essence not only seems to be clear, 
but this requirement also considerably restricts what can be called an ‘image’. 
Setting aside this apparently contradictory claim, what else can we learn 
about Scholz’s concept of image games? 

Scholz claims that there is a vast variety of different image games (see ibid., 
373). Contrary to Wittgenstein, however, he brings in a level of systematising 
by distinguishing between two different kinds of image games. These two kinds 
are characterised by their different directions of fit. On the one hand, there are 
image games that imply for their pictorial instances a fit from world to image. 
Examples are blueprints in architecture or in the engineering sciences (see ibid., 
371). On the other hand, there are images that are directed at the world, that is, 
images that mimic previously present objects of the real world (see ibid., 372). 

Two further aspects are of prominent importance with respect to image 
games. Firstly, Scholz makes clear that one particular image can be used in 
very different games. For instance, a photograph of a dog can be shown either 
to inform someone else about what the photographer’s pet looks like or it 
can be meant as a warning, for instance when attached to the fence of the 
photographer’s garden (see ibid., 374). Consequently, in order to understand 
an image correctly, we have to know in which image game it is employed. As 
Scholz points out (see ibid., 377), particularly regarding a correct historical 
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understanding of certain images, this becomes extremely relevant in addition 
to background knowledge about the cultural specifics and practices. 

Secondly, and as a consequence of this close connection between purposes 
of usage and meaning, there is (or can be) a distinction between the context 
of producing an image and of its subsequent usages (see ibid., 372). This dis- 
tinction implies not only a change in meaning due to the different purposes 
involved in using the image at hand, but also that completely different indi- 
viduals might interact in both contexts. Thus, many more agents can be 
involved than a simplistic focus on the context of image production alone 
suggests. 

Now, Scholz thinks that many image games are related to communica- 
tive purposes (see ibid., 373). I would maintain, rather, that a// of them are 
somehow related to communication. This point of view is a consequence 
of how the term ‘communication’ is understood. I prefer a broad concept 
here, obviously broader than the one used by Scholz. The concept used here 
is based on ideas propounded by Paul Watzlawick and his colleagues. They 
suggest that all kinds of behaviour in social settings can be interpreted as 
communicative acts and that, as we cannot stop behaving, we will never 
cease communicating in those contexts (see Watzlawick, Beavin and Jackson 
1974, 51). 

Although Scholz relies on Goodman’s theory of depiction, which has 
already been pointed out as being unsuitable in the context of scientific 
images, his concept of image games is nevertheless very useful in the attempt 
to clarify the role of visualisations in scientific communication and, in par- 
ticular, in scientific argumentation. The theory of image games makes clear 
that there are more aspects to consider when trying to understand what is 
meant by a particular image in a communicative context than its encoded 
content alone. 

Leo Groarke’s conceptual theses about argumentation pave the way 
for further analysis. He claims that “‘[a]rgumentation’ includes not 
only arguments, but also the broader dialogue, discussion and disagree- 
ment in which arguments are embedded. Acts of arguing attempt to 
rationally establish some conclusion by providing evidence in its favour” 
(Groarke 2015, 135). Thus, regarding argumentation as a communica- 
tive act!” permits utilising the theory of image games and, hence, disen- 
tangling correctly what has been called a ‘layering of meanings’. Such a 
layering is the result of a multiple usage of one particular image. In such 
a case, there is the genuine meaning encoded into the visual represen- 
tation by its producer. This basic meaning, however, will be covered (at 
least partly) by layers of meaning superimposed on the genuine one by 
subsequent users. Making use of visual representations in image games 
can, at least, add a second meaning to what has genuinely been encoded 
into the visual format, and this second meaning might change due to the 
different games played with the aid of this particular image. A compilation 
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of meanings, acquired in different image games, might also be possible, but 
would be rather an exception than an example of common practice. 

Nevertheless, adding layers of meaning in image games to what the visual 
representation was genuinely supposed to express by its producer is the 
reason that Scholz urges us to reconstruct the genuine image game and cul- 
tural surroundings if visual representations are to be correctly understood. 
Important dimensions of meaningfulness would simply be overlooked if the 
focus were solely on the context of production. Hence, the theory of image 
games permits discovering what additional levels of meaning images can 
acquire by being used in argumentation. The question of how exactly images 
play a role in argumentation is answered in the second part of the analysis, 
which is concerned with ways to deconstruct the act of argumentation to see 
if and how visual representations contribute to its different parts. 

Kjorup makes us aware that understanding visual representations correctly 
has to take into account the whole communicative act, not the image in iso- 
lation (see Kjorup 1978, 57). Moreover, by taking speech act theory as the 
relevant paradigm, he makes clear that pictorial speech acts are composed 
of different parts, namely an illocutionary, a locutionary and the propos- 
itional act (or content, see ibid., 61). Taking these levels into account and 
adding the perlocutionary act offers us the means to show at what levels visual 
representations can make contributions to scientific arguments. 

To begin with, the significance of these sub-acts in speech act theory should 
be briefly summarised: the locutionary act concerns the uttering of words 
or, with regard to visual representations, the showing of images in conversa- 
tional contexts. The illocutionary act is about the kind of act performed, such 
as warning somebody by showing the image or simply informing her of the 
appearance of what is depicted, etc. The propositional act is about encoding 
the basic information, for instance by drawing a certain figure or taking a 
photograph (see ibid.). And finally, the perlocutionary act is about the effect 
intended on the recipient, for example, evoking a particular emotion or con- 
vincing the recipient of a certain proposition. 

Kjorup tries to spell out rules for all of these sub-acts when images are 
utilised in communicative contexts (see ibid., 61ff.). He points out that the 
context of usage 1s important both to fix the reference of a particular visual 
representation (see ibid., 58) and to determine what particular illocutionary 
act is performed by using a certain image (see ibid., 65). Even a photograph 
seems to presuppose some background knowledge about what is depicted in 
order to discern exactly to what it refers. For instance, it is only by reading the 
caption accompanying Figure 2.15 that we come to know that it shows lava 
flows on Olympus Mons on Mars. Without this additional information, we 
would most probably be unable to interpret to what this photograph refers. 

Moreover, Kjorup discusses particular illocutionary acts that can be 
performed with the aid of visual representations. For example, he tries to 
establish the rules governing the illocutionary act of “illustration” (see ibid., 
66ff.), which is particularly interesting with respect to the intertwining of text 
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and image that plays a major role in this case in order for it to be successfully 
performed. 

Applying these ideas to the broader context of scientific argumentation, it 
seems reasonable to assume that argumentation can be similarly subdivided 
into particular acts, as suggested above with respect to speech acts — although 
one might object that arguments are composed of assertions and that it is this 
kind of act that should be analysed when discussing the nature of arguments. 
Correct though this might be, the relevant level of analysis still seems to be 
about arguments and not about assertions. Moreover, to a certain degree it 
is also possible to transfer the subunits of speech act theory to arguments. 
How does this work and why is this of relevance when theorising about visual 
representations in scientific arguments? 

I shall start by investigating the possibility of transferring the units of ana- 
lysis to the context of argumentation. When arguments have to be uttered or 
otherwise employed, a locutionary act occurs. Arguments entail certain infor- 
mation that the speaker will encode them in a certain way — the propositional 
act. They are meant to transmit information by offering reasons to believe in 
a certain conclusion supported by them that could be called the illocutionary 
act. This transmission of information can then be connected to the various 
intentional aims of the speaker. She might want to transmit knowledge, she 
might want to ponder on the consequences of certain hypotheses herself, or 
she might want to solve a particular dispute. This last option, for example, is 
put forward by Groarke as the aim of argumentation. He attempts to clarify 
the nature of ‘arguments’ by writing that: 


I define an argument as a standpoint (a conclusion) backed by reasons 
(premises) offered in support of it. In a typical case of arguing, arguments 
are an attempt to resolve disagreement [...], though they may also function 
as an attempt to avoid disagreement by securely establishing some belief. 
I will understand an act of arguing as an attempt to use premises and 
conclusions to resolve some disagreement or potential disagreement. 
(Groarke 2015, 134f.) 


Finally, arguments are supposed to convince the recipient about what has 
been transmitted by argumentative means; in this case, this could be regarded 
as the perlocutionary act. This subdivision of arguments is relevant in the 
current context because quandaries about cases of using visual representations 
as parts of scientific arguments can apparently best be understood as a con- 
sequence of their contributing to different sub-acts of argumentation. Making 
use of visualisations in the context of scientific argumentation does not neces- 
sarily imply that they are meant to entail information relevant to the argu- 
ment at hand, that is, they do not have to be parts of the propositional act. 
They can also be used as a means to support the perlocutionary act. Clearly, 
visualisations are used to support the persuasive power of arguments in many 
instances. This functional role can be correlated with transmitting additional 
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information such as evidential data. Visual representations might also be used 
to present data in an organised way that makes correlations salient. However, 
particularly in popular science, visualisations simply might also be used 
because of their persuasive and emotional power and, in this sense, have to be 
evaluated cautiously if the epistemic dimension is of interest. 

Finally, it can be asked whether visualisations can also be used as means 
to perform the locutionary act. Is it possible to ‘utter’ an argument simply by 
showing an image? Clarifying this question seems to be particularly relevant in 
contexts where the contending parties introduce images in an ongoing debate in 
which, say, party Y is urged to counter an argumentative move made by party 
X by also making use of images to support her position. A telling example 
of this kind is discussed by Randall A. Lake and Barbara A. Pickering. Their 
case study is about the debate between anti-abortionists and pro-abortionists 
in the United States (see Lake and Pickering 1998) and will be discussed in 
some detail below. Lake and Pickering’s account will be discussed in some detail 
below. For the moment it will suffice to note that their example makes it particu- 
larly clear that there are instances where contending parties seemingly have no 
other choice than to make use of visual representations in their argumentation. 

Let us now come to the third part of my analysis, namely to the question 
of whether visual representations can also play a part with respect to trans- 
mitting propositional contents. Groarke, for example, approaches this topic 
rather carefully by claiming that: 


[t]he kinds of examples that motivate me are acts of arguing that involve 
pictures, maps, sounds, diagrams, smells, video clips, and other non-verbal 
phenomena which are not propositional in the way that verbal statements 
are (though it bears noting that the relationship between sentences and 
propositions is itself a matter of much controversy). 

(Groarke 2015, 135) 


Obviously, he thinks that non-verbal elements can also function as premises 
and conclusions in argumentation. However, he does not try to pin down the 
claim that these non-verbal elements are propositional in the sense claimed 
for sentences. 

A different approach is suggested by Lake and Pickering. They examine 
the possibility of visual arguments by taking a look at films. They do not 
want to analyse arguments of mixed media, but images as exclusively visual 
arguments. At least, this is what they attempt to investigate, although in the 
end they admit that their case study actually shows results only for a “mixed- 
media environment” (Lake and Pickering 1998, 91) — which is not surprising 
since they focus on films as their object of research. However, what Lake and 
Pickering might regard as a shortcoming in their analysis becomes a virtue in 
the context of our current discussion. 

In their analysis, they try to show in what sense images can be said to refute 
one another. The authors start with the assumption that visual representations 
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do not have a propositionally structured content (see ibid., 81). Moreover, 
they regard the ability to refute as essential to whatever kind of representation 
is used in argumentative contexts. In contrast to Perini, Lake and Pickering 
deny that images can be bearers of truth values, but they nonetheless think that 
images can be used in argumentative contexts since they can refute each other. 

They point out three different ways how such a refutation can be achieved 
by visual means, namely: 


(1) through dissection, in which an image is ‘broken down’ discursively, 
its component parts named and its relations analyzed, thereby opening 
the image to refutation via traditional (discursive) argumentative 
means; (2) through substitution, in which one image is replaced within a 
larger visual frame by a different image with an opposing polarity; and 
(3) through transformation, in which an image is recontextualized in a 
new visual frame, such that its polarity is modified or reversed through 
association with different images. 

(Lake and Pickering 1998, 81f.) 


Lake and Pickering discuss these assumptions by analysing films used by 
opposing parties in the dispute about abortion in the US. They show how 
anti-abortionists and people defending women’s rights regarding abortion 
make use of different films that become part of the debate and are used in 
such a way that these films can be said to refute one another. 

In particular, they discuss how the anti-abortionist film The Silent Scream 
(SS) is dissected within another film (A Planned Parenthood Response to 
‘Silent Scream’ (PPR)) produced by the contending party. This is meant 
as an example of the first strategy of refutation. For instance, the plastic 
model used in SS to demonstrate the developmental state of the foetus is 
criticised as inappropriate in scale. Furthermore, the slow-motion technique 
used in S'S to create the impression of a very calm and secure environment 
for the foetus in the womb is highlighted as a manipulative means in PPR. 
They make clear that the SS film-makers used slow motion to manipulate 
the recipients’ impressions by visually demonstrating the frame-by-frame 
changes visible in their film (see ibid., 86). Beyond that, PPR also makes use 
of the strategy of substitution. Whereas SS focuses on the foetus as a victim 
and presents pregnant women suffering passively after abortions, PPR shows 
women as rational and active decision-makers. “It replaces images of women 
as passive and pregnant with images of women as active professionals” (ibid., 
87). Finally, PPR also offers an example of the last strategy of refutation, 
namely transformation. Key images of SS are presented here in a different 
visual frame. For example, a specialist is shown watching SS on a TV screen, 
commenting critically on what is shown (see ibid.). 

What becomes clear in Lake and Pickering’s analysis is that participants 
in a controversy not only have to pay attention to their opponents’ verbal 
statements, but also have to operate in a visual way if their rivals make use of 
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visual representations in the course of the debate. One reason for this obvious 
necessity might be found in the commonly stated persuasive power of images.'® 
Lake and Pickering offer a much stronger reason when indirectly referring to 
the evidential role of visual representations. Interpreting the anti-abortionists’ 
communicative intention to use films in the debate, they conclude that for 
these people films even have the status of proofs.'? At the end of their paper, 
Lake and Pickering state that: 


[iJnterestingly, SS (and anti-abortion rhetoric generally) implicitly adopts 
the view [...] that pictures cannot argue, and adapts this view to ideo- 
logical ends; that is, it contends that images of the fetus are beyond argu- 
ment and constitute irrefutable proof of the fetus’ humanity. 

(Lake and Pickering 1998, 91) 


More precisely, the thesis that is supposed to be proven visually by SS is that 
“the fetus is an unborn ‘child’ and that abortion therefore must be murder” 
(ibid., 84). Obviously, Lake and Pickering regard ‘argumentation’ as an 
exchange of reasons in this context. If one side achieves for its thesis the 
status of a proof, a continuing exchange would make no sense, as apparently 
no question is left for further discussion. 

Lake and Pickering contrast this claim about what anti-abortionists 
attempt to achieve by showing their film with what the authors think is the 
underlying assumption of the second party’s way to make use of films in this 
context. 


In contrast, PPR implicitly adopts the view [...] that pictures can argue, 
and adapts this view to contrary ideological purposes; that is, it contends 
that images of the fetus are only argumentative, are susceptible to refuta- 
tion, and constitute misleading evidence of the fetus’ humanity. 

(Lake and Pickering 1998, 91) 


Lake and Pickering elaborately show different strategies of utilising films — 
and thus of visual representations — in an already highly charged emotional 
debate. They point out the dynamics of the debate that played a role in pro- 
ducing and making certain visual representations popular. In particular, it 
becomes clear that, because of the persuasive power of images, the contending 
parties were simply not able to ignore this shift in representational means 
in the debate and stick to an exchange of linguistic arguments. They had to 
bring into play something that could deal with this persuasive capability of 
images. Doing so, however, does not necessarily imply that in such instances 
pathos is given priority to logos, as critics usually object. Contrary to this, 
Lake and Pickering point out that refuting the visual representation of The 
Silent Scream also means presenting a detailed and careful analysis of what is 
shown in this film and making clear how and why those images were produced. 
These analytical steps are accompanied by visualisations that are meant not 
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only to support the verbally but also the visually presented explanations of 
experts (see ibid., 86). For example, the expert’s claim that it is neurologically 
impossible that the foetus screams inside its mother’s womb is supported by 
images showing the development of the human brain. At the relevant stage, 
it is neurologically impossible that the brain of the foetus is able to register 
alarm (see ibid.). This illustrates the point that making use of images in argu- 
mentation does not necessarily imply dramatisation and appeals to emotional 
stimuli. They can also be used in an explanatory and/or evidential way. 

Beyond that Lake and Pickering’s analysis also shows how images can 
make contributions to the different sub-acts of argumentation. 


e Visualisations were necessary to utter the argument (locutionary act). 
The examples in this context are films. Moreover, visual representations — 
such as the ultrasound images of the foetus, or images of the develop- 
mental stages of the foetus’s brain — are displayed within this audiovisual 
medium. 

e Images were used as persuasive means (perlocutionary act). For example, 
the ultrasound images of the apparently screaming foetus and its apparent 
attempts to avoid the abortion are supposed to provoke feelings of sorrow 
and compassion amongst the recipients. 

e Visual representations entail certain information (propositional act). The 
PPR film-makers, for example, demonstrate visually that their opponents 
produced a propaganda-film making secret use of slow-motion techniques 
to provoke certain emotions. 

¢ Finally, images were brought into play to securely establish some belief — 
to use Groarke’s words (see Groarke 2015, 135) — (illocutionary act). By 
showing their films, both parties try to convince an audience of their 
respective opinion, namely that abortion is either murder or that it is not. 


What remains to be shown is how to deal with the thesis that visual 
representations can provide propositional knowledge, although it is disputed 
by traditionalists that images are propositionally structured. Two further 
proposals of how to conceive of visualisations in the argumentative context 
should therefore be examined more closely at this point. 

To begin with, (1) Axel Arturo Barceló Aspeitia suggests taking them 
as instances of what he calls “heterogenous arguments, i.e. arguments that 
are not conveyed through a single medium, but instead make use of both 
verbal and visual resources” (Barceló Aspeitia 2012, 356). He thinks that, 
in these contexts, visualisations can indeed make proper contributions 
which “can be either sub-propositional (i.e. properties and functions that, 
properly combined with information conveyed through other means, like 
words, or available in the context, can yield full propositions) or fully prop- 
ositional” (ibid.). Thus, Barceló Aspeitia is convinced, like Perini, that 
visual representations can entail propositionally structured information. 
Contrary to her conviction, however, he does not claim that images can 
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be bearers of truth values. By analogical reasoning, he compares heter- 
ogenous arguments with sub-sentential ones. The latter contain fragmentary 
statements, either as premises or as conclusions, and are nonetheless under- 
stood as transmitting full propositions (see ibid., 357). In such cases, the 
context yields the missing information. This contextual information enables 
the hearer to understand what was meant by the sub-sentential argument. 
Barceló Aspeitia thinks that, just as language can be used in a fragmentary 
way to transmit complete propositions, images can also play this functional 
role. Thus, he tries to show how, in certain instances, image and text work 
together to transmit the information intended by the speaker or writer. 

Barceló Aspeitia points out that it is common practice to use a combin- 
ation of different modes of representation — or media, as he calls them. The 
examples he mentions are wanted signs, store catalogues and advertisements. 
We do not usually have any difficulty understanding that the combination of 
an image and a numerically expressed price tells us that a certain item is meant 
to be sold for a certain amount of money (see ibid., 360). In this way, image 
and text interact to complete the message. 

Yet it might be objected, as Barceló Aspeitia points out, that this infor- 
mation transmission is only possible if the proposition in question is 
reconstructed verbally by the respective recipient (see ibid.). Contrary to such 
an assumption, however, he defends the thesis that paraphrasing is not neces- 
sary. As a rationale for this claim, he refers to the difficulty of translating 
the non-verbal elements in a precise and unambiguous manner. He concludes 
that “[p]recisely because it is not always possible to translate heterogenous 
arguments into verbal ones, it is very unlikely that that is what happens every 
time we interpret heterogenous arguments” (ibid., 361). Nonetheless, the mere 
possibility of highlighting certain translations as more adequate than others 
and, furthermore, of ruling out certain sentences as completely inadequate as 
translation demonstrate, from his point of view, that visual representations 
convey propositional content — that is, a content being used as a benchmark 
to assess the adequacy of certain translations. 

An example is provided by the successful detection of gravitational waves 
mentioned in the introduction to this book. A crucial part of the argument 
that a successful detection had been achieved was that two measurement 
devices registered the signal independently and thereby raised the statis- 
tical significance of the evidential status of the event registered. This claim 
is supported by the fact that the two diagrams (see Figure 1.2) showing the 
registered data match to a significant degree. This seems to be the usual way 
to translate visual representations, such as the diagrams in this example, in the 
context of a heterogenous argument. However, the fit between both diagrams 
claimed here is doubtless only a rough approximation to what is visually 
conveyed. Yet the conflation of the two diagrams also shows that a claim such 
as “The LIGO project at Hanford registered completely different data from 
the LIGO project at Livingston” would not be an admissible translation of 
these diagrams. 
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Barceló Aspeitia comes to a positive result when considering the question 
of whether visual representations can transmit propositional content. 


Thus I conclude that the contribution images make to this kind of argu- 
mentation is substantial and direct: exploiting information from the con- 
text, they provide information necessary for the communication of the 
propositions that play the roles of premises and conclusion. Furthermore, 
they achieve this directly, without the need of verbalization. 

(Barcelo Aspeitia 2012, 365) 


Yet contrary to some conceptions of ‘proposition’, he also defends the thesis 
that propositions are not linguistic entities (see ibid.). They can be trans- 
mitted linguistically, but this can also be achieved by using other kinds of 
representations. Thus, what alters the scales here is the question of how 
exactly to conceive of ‘propositions’ — and Barcelo Aspeitia at least partly 
removes this concept from an assumed close connection to verbal expressions. 
That is, his considerations undermine the assumption of a need to express 
propositions in verbal statements right from the start, or at least the possi- 
bility of translating the content transmitted by other representational means 
completely into verbal statements. This interesting move will be scrutinised in 
more detail in due course when analysing the nature of the cognitive content 
of visual representations below. 

The second strategy (2) to deal with visualisations in argumentative contexts 
that will be discussed here is Groarke’s approach to the topic. His proposal 
about “multimodal arguments” offers a different perspective with regard to 
the relevance of propositionality in the context of argumentation (see Groarke 
2015). He agrees with Perini and Barceló Aspeitia that visual representations 
can be regarded as proper parts of arguments. Furthermore, Groarke defends 
the claim that these parts do not have to be propositionally structured to play 
the epistemic roles they are supposed to fulfil in those contexts (see ibid., 135). 
He attempts to broaden the concept of argumentation in an even more wide- 
ranging sense than, for example, Barceló Aspeitia. He writes: “[t]he classical 
model of argument understands arguments to be entities made up of words 
and sentences. In using other modes of arguing, arguers build arguments from 
other kinds of ingredients — pictures, diagrams, non-verbal sounds, tastes, and 
so on” (ibid., 140). Thus, Groarke regards a broader class of “ingredients”, 
as he calls them, as possible candidates in argumentative structures. As a 
result, his conception by far exceeds my attempt to show how the three modes 
of representation — linguistic, visual and numerical — interact in the context 
of scientific argumentation. Groarke’s broader perspective, moreover, also 
permits the detection of an even closer connection between visual representa- 
tion, observations and sense perception as an epistemic source. 

What exactly does Groarke suggest? Having pointed out that there are 
a lot of different “ingredients” a speaker might utilise in her argument, 
he proceeds by defining different modes and sub-modes of an argument 
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along the lines of these ingredients. In order to analyse these multimodal 
arguments, he suggests the usage of a “Key Component” (KC) table (see 
ibid., 135). This table summarises the different components used in a par- 
ticular argument by displaying in a first column “acts of arguing” (actions 
such as the utterance of claims, the display of visual data, etc.), the struc- 
tural parts of the argument (premises and conclusion) in a second, and, 
finally, in a third column the mode used to express the different parts (visual, 
verbal, etc.). Although Groarke admits that using such a table to analyse 
multimodal arguments also means interpreting the different components 
to some extent (see ibid., 139), this method nonetheless permits a more 
faithful reconstruction than does a mere verbal translation of the relevant 
parts. Groarke’s suggestion seems to be a useful addition to the traditional 
methods of argument-reconstruction in verbal forms employed to analyse 
their validity and conclusiveness. 

Even more importantly, however, he points out that not only visual 
representations but also actual visual demonstrations can be parts of argu- 
mentation (see ibid., 148f.) — respectively, components that we perceive by 
our other senses such as smells or sounds (see ibid., 149f.). All of them are 
instances of using perception as an epistemic source. In order to better under- 
stand in what sense it can be said that these instances contribute to argu- 
mentation by adding information to premises and conclusions, the epistemic 
capacities of perception should be examined more closely. How do they 
work and what kind of knowledge (propositional or non-propositional) do 
they yield? 

To sum this up, both, Barcelo Aspeitia and Groarke try to pave the way for 
a more moderate conception of argumentation which would justify Perini’s 
observation that it seems to be common practice to make use of other (in par- 
ticular visual) components in the context of scientific arguments. Nonetheless, 
both accounts also raise the question of how (and what kind of) information 
is extracted from these different components to be employed in argumenta- 
tion. This leads to a discussion of perception as an epistemic source and is 
closely connected to the question about what kind of content we cognitively 
access when deciphering information encoded in visual representations. Both 
questions will be considered in section 4.2. 


4.1.4 Interim results: what can be learnt from argumentation theory? 


In the previous sections, a closer examination was made of the possibility of 
visual arguments. This term is used in the debate to refer both to arguments 
that contain visual representations as (parts of) premises or conclusions, 
and to images that are supposed to express a complete argument. Perini has 
pointed out the tension between, on the one hand, scientists’ practices of 
making extensive usage of visual representations in epistemic contexts of this 
sort and their obvious conviction that visualisations can undoubtedly fulfil 
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the epistemic tasks they are supposed to perform and, on the other, the ten- 
dency of philosophers to ignore them or to hold them to be epistemically 
impotent. 

It was argued above that this ignorance is at least partly a result of the 
linguistic turn at the beginning of the developmental history of analytical 
philosophy. This focus on language still dominates studies of this discip- 
line and hence approaches connected to this field, such as the philosophy 
of science. In particular, arguments — that serve, amongst others, as means 
to transmit and defend scientific hypotheses — are typically regarded as lin- 
guistic entities. Moreover, as the discussion of this traditional point of view 
has shown, the emphasis on verbal expressions is also related to the question 
of what kinds of representations are suitable candidates in logical operations. 
In particular, entailment relations and the capacity to bear truth values are of 
relevance when it comes to (scientific) arguments. Both features, however, are 
associated with linguistic expressions in the Fregean tradition. Thus, visual 
representations are dismissed in this context as apparently belonging to the 
wrong kind of representation. 

At best, people defending the traditional point of view in philosophy 
would call for a translation of information potentially contributed by 
visual representations to linguistic arguments. Thus, they think that proper 
contributions are only possible via a paraphrasing — a method that is supposed 
to take place in the recipient’s mind — if it is not made explicit in the relevant 
publication. It is interesting to note that the demand of such a paraphrasing into 
linguistic expressions is not restricted to the realm of scientific argumentation. 

Exactly the same sort of discussion can be found, for example, with respect 
to the epistemic functions of thought experiments (see e.g. Brown and Fehige 
2011, sect. 3.2). Here, too, the necessity to paraphrase the content presented 
visually is discussed to make use of it within the epistemic process in which 
it appears. With regard to thought experiments, this similarity to the above 
discussion comes as no surprise because the approach defended by John 
D. Norton in this respect is simply that “thought experiments in science are 
merely picturesque argumentation” (Norton 2004, 1142, my italics). Thus, 
Norton not only draws an analogy between the epistemic function of thought 
experiments and scientific arguments, he actually affords them parity. From 
his point of view, arguments can replace thought experiments without losing 
essential information (see Norton 1996, 336). In developing this thesis, he tries 
to make a case against James Robert Brown’s ideas on the topic. Contrary to 
Norton, Brown thinks that the picturesque element of thought experiments is 
not simply reducible to (verbal) arguments. He states that thought experiments 
work epistemically because they are visual in certain respects. In particular, he 
emphasises the relevance of perceptually accessing the content of thought 
experiments. Of course, such a conception presupposes certain hypotheses 
about the nature of phenomena, their correlations (laws of nature) and the 
way these essential facts are understood. 
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To start with, from an ontological point of view, Brown defends a Platonist 
approach. That is, he thinks that abstract objects, such as mathematical 
objects, exist independently of human beings and do not belong to the realm 
of time and space. Moreover, he assumes that laws of nature are relations 
between properties. These relations are regarded as abstract objects as defined 
above (see Brown 2004, 1130f.). But how can we grasp these laws of nature? 
This is the point where it is of relevance that thought experiments are visual 
entities. Brown is convinced that we are able to perceive the laws of nature 
when visually engaging in thought experiments. He writes: 


[a]Jccording to mathematical Platonism we can perceive the abstract 
entities of mathematics. Not all, of course, but we do have intuitions 
of some of them. So, it’s possible to perceive abstract entities, at least 
some. Usually we learn laws empirically, by seeing instances. But laws are 
abstract entities, so they could be perceivable, too. How could we have an 
intuition of a law of nature? The obvious thing to conjecture is that we 
grasp them via thought experiments. Laws and numbers are both outside 
of space and time. If we can see one, then we should be able to see the 
other. Thought experiments are telescopes into the abstract realm. 
(Brown 2004, 1131) 


Translating thought experiments into verbal arguments would then imply 
cutting off the ability to epistemically access the relevant content that they 
reveal to the recipient. At least our understanding of the relevant correlations 
would be hampered to a certain degree. 

Brown’s ideas on the topic undoubtedly deserve closer attention in order 
to understand correctly what is implied by his different hypotheses. In par- 
ticular, his approach to regarding thought experiments as perceptual tools 
to grasp laws of nature as abstract objects is obviously not an easily handled 
topic. Consequently, the discussion of this account here is admittedly rather 
brief, as I will not go into the details of Platonism; nonetheless, the contro- 
versy between Brown and Norton is worth mentioning in the context of the 
current discussion. It makes explicit an important question already present 
above in the discussion of visual arguments in science, namely what a trans- 
lation of possible contents of thought experiments into (verbal) arguments 
would imply. 

The question about the consequences of a translation reveals different 
dimensions of the topic that should be kept apart. (1) If a translation is pos- 
sible, then there is a cognitive content of some kind present in both instances. 
Are there particular difficulties with regard to such a translation? If there 
are, this might mean, in a negative sense, (2) that one of these representa- 
tional means is inferior to the other in performing a certain epistemic task. 
Ambiguities, for example, might cause problems in translations. If precision 
is required in the situation at hand, representations that do not meet this 
standard seem to be inappropriate tools to serve the respective purpose. This 
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also shows that (3) evaluations of inferiority and superiority are not abso- 
lute. They are relative, depending on the context of usage and the task to be 
performed with the aid of representational means. Nonetheless, there might 
be contexts (4) in which our perceptual sense is of particular use in grasping 
certain information, as evolution has emphasised its development in human 
beings. Vision is our primary sense to perceive, to cognitively access the world. 
It should not come as a surprise that images, which allow us to make use of 
our visual sense to decode them, are easier to handle in the epistemic pro- 
cess than other modes of representation. Finally (5), there might be certain 
aspects that can be accessed via perception only (this is what seems to lie at 
the heart of Brown’s ideas about thought experiments) and thus undermine 
the suggestion of translating visually presented information into verbal infor- 
mation in any possible context. 

These five theses constitute the stepping stones to the next questions. Do 
visual representations contain a cognitive content? And, if so, is it similar to 
the propositional content of verbal expressions? If it is stmilar, this might pro- 
vide a clue to rethinking the concept of propositions as currently in use in the 
philosophical community. The tight linkage to linguistic expressions may have 
to be considered anew because propositional content can also be transmitted 
by visual means. However, if it turns out that, although the content entailed in 
visual representations is not similar to the nature of the alleged propositional 
content in the Fregean tradition, it can nevertheless be shown that at least 
some visual representations are epistemically potent, this might then be taken 
as a clue to reconsider the thesis that epistemic usefulness is exclusively tied to 
propositional content. 

The discussion of these two alternative approaches on how best to explain 
the epistemic potentiality of images in the context of scientific arguments 
naturally takes its course by analysing perception as an epistemic source. 
It is through perception that the content of visual representations is cogni- 
tively accessed. Clarifying this route of access will thus also contribute to an 
understanding of how visual representations may be epistemically effective. 
This, then, is what will be analysed in more detail in the following section. 


4.2 The cognitive content of visual representations 


Making use of visual representations in scientific arguments in the way Perini 
explains this process, namely that scientists “describe figures as supporting, 
or as expressing, the conclusion of an article” (Perini 2010, 134), implies a 
particular attitude towards those representational means. Scientists seem 
to assume that those images entail certain information and, additionally, 
that they are able to transmit this information in an appropriate way. These 
attitudes towards visual representations in science are one of the main reasons 
supporting the initial claim that scientific images can best be regarded as signs. 

Perini takes this observation about communicative practices in science 
and the place of visual means therein to demand that “[w]e [i.e. philosophers 
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of science, N.M.] need to take a more fundamental approach to understand 
what kind of contents visual representations convey, and how they do so” 
(ibid., 135). Thus, there is a need for “a more specific account of visual 
representations as communication tools — a theory of signs or a semiotics of 
visual representations” (ibid.), as she expresses it. In this section, an approach 
to account for this capability of visual representations to serve as communi- 
cative tools and will proceed as follows. 

Firstly, a start can be made by focusing on the capacity of visual 
representations as content providers in scientific arguments. It is necessary to 
analyse in more detail in what sense this capacity can be accounted for and 
what exactly the nature of the content is that can be transmitted via visual 
representations. This topic is connected with considerations about trans- 
latability and the limits to what is expressible by certain representational 
means. Certain theses of this sort have already appeared in section 2.2 of 
this book. The consideration now must be why certain characteristics of 
visual representations do not apparently preclude the latter from being used 
in scientific communication, although philosophers tend to regard them as 
shortcomings. 

Moreover, taking seriously the possibility of translations from one repre- 
sentational means into another raises the question of whether this transfer 
of information only works in some directions but not in others. Taking into 
account the initial distinction made earlier between numerical, linguistic 
and visual representations, it might be argued that certain translations will 
imply a loss of information, which might lead us to the assumption that 
one of those representational means is more ontologically and epistemo- 
logically basic than the others. That is, although all information can be 
encoded in a (visual) representational means when transferred into another 
medium, the content provided might lose particular pieces of information 
in that process. In section 3.1.1 of this book, we have already come across 
the puzzle that digital images present when discussing the epistemic status 
of scientific visualisations. With regard to digital images, the question arose 
of whether numerical data should be considered as the more basic kind of 
representation, as, obviously, digital images are rendered in such data sets. 
In the following, such an apparent reducibility thesis will be examined more 
thoroughly. 

Closely connected to this issue of reducibility is the question of how 
exactly to conceive of the content of visual representations in science, that is, 
the question of whether it is propositionally structured or not. What links this 
to the discussion of reducibility is the following consideration: although there 
are many different accounts of what the term ‘proposition’ exactly means,”! 
one aspect that is commonly accepted is that propositions are expressible by 
linguistic sentences. At least in this way, propositions are tied to language, 
although they themselves do not necessarily have to be linguistic in kind. 
A corollary of this is that if the content contained in and transmitted via 
visual representations is non-propositional in certain respects, it cannot be 
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translated and thus is not reducible to linguistic expressions without a loss of 
information. 

Secondly, the topic of perception as an epistemic source will be discussed in 
order to come to tackle this question about the assumed propositional or non- 
propositional content of scientific images. Obviously, the information that is 
presented via visual representations has to be perceived, that is, their content 
has to be cognitively accessed by vision — the primary human sense. It does 
not therefore seem far-fetched to assume that the epistemic mechanisms of 
perception will also disclose something about the special case of image per- 
ception, i.e. the decoding process of information presented visually. A closer 
look at the epistemological debate about perception in general thereby reveals 
an interesting fact concerning the question of propositions. On the one hand, 
contemporary philosophers seem to have no particular problem in regarding 
perception as an epistemic source, 1.e. as a source of propositional knowledge. 
On the other hand, there is no consensus about the content of perception 
and how exactly it features in processes of justification concerning perceptual 
beliefs.” This presents a difficulty similar to the case of visual representations 
in science. Although knowledge claims based on visually transmitted infor- 
mation (either via perceiving images or by perceptual experiences of other 
entities in the world) are commonly accepted, there is a dispute over whether 
the content of images is propositionally structured or not. Contrary to the 
case of everyday perception, this latter question is employed to undermine 
claims concerning the epistemic capabilities of visual representations in com- 
municative contexts, whereas a potential non-propositional structure of per- 
ceptual experience in general is not usually indicated as a particularly serious 
obstacle to gaining knowledge via perception. This discrepancy between the 
two cases seems to be worth closer investigation. 

It can be assumed that this difference is a consequence of the initially 
defended thesis that scientific images can best be regarded as signs. Taking 
this claim seriously, one might object that, particularly in the context of sci- 
entific communication, those signs cannot simply be perceived in order to 
decipher their content, but that background knowledge is necessary to inter- 
pret them correctly. It might be argued that permitting the thesis that non- 
propositional content plays a significant part in visual discourse would clash 
with the communicative aims of transmitting precise information and clear 
messages. The compatibility of the claims that scientific images are signs and 
that perception is the primary way to decipher their content will therefore be 
analysed thirdly. Before discussing these difficulties, a start can be made by 
scrutinising the content characteristics of visual representations in science in 
more detail. 


4.2.1 Content translatability and the reducibility thesis 


In this section, the question will be investigated whether it can be stated 
that visual representations possess a content that works similar to Fregean 
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Figure 4.3 Image of Pluto’s vast, icy surface. This part of its surface is informally 
called Sputnik Planum. The image was taken from a distance of 50,000 
miles (80,000 kilometres) as New Horizons flew past Pluto on July 14, 2015. 


Source: Southwest Research Institute, www.nasa.gov/image-feature/image-of-plutos-vast- 
icy-plain-informally-called-sputnik-planum 


‘thoughts’ in linguistic expressions, that is, whether they have a content that 
human observers can cognitively access and process to gain knowledge or not. 
My approach to this topic is via an analysis of content translatability from 
one representational means to another. An example will illustrate what is at 
issue here. 

In 2015, after a nine-year voyage, the NASA spacecraft New Horizons? 
finally reached its first mission destination, the dwarf planet Pluto and its 
system of satellites. Since its arrival, the space probe has started taking and 
transmitting photographs of Pluto (see Figure 4.3) and its satellites. Of 
course, these are digital images, not analogue photographs. What is trans- 
mitted to Earth are digital data that allow the corresponding image to be 
rendered when the data have been received. In this case, a translation between 
the numerical and visual mode of representation apparently works without a 
loss of information. 

This kind of translatability is not very surprising, because digital pho- 
tography works by storing information detected by a charge-coupled device 
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(CCD) sensor in binary units that allow the reconstruction of visual informa- 
tion as a picture later on. Interestingly, this process also works in reverse. That 
is, starting with an analogue image, we can transfer it into bits and bytes by 
simply scanning it, and it is not only images that we digitalise in this way but 
texts, that is, written language, as well. Moreover, oral language can also be 
recorded and digitally stored. Thus, switching back and forth between numer- 
ical data on one side and all other modes of representations on the other 
seems to be fairly unproblematic. Again, this is not very surprising as this is 
how information-technology devices work. 

Regarded in this way, then, no representational means seems to be more 
basic than numbers, or rather ‘digital units’, as even numbers are reduced 
to bits in IT contexts, that is, to ‘basic units of information’. These units are 
expressed by binary representations such as the commonly used 0-1 schema. 
However, they could also be represented quite differently. 

Now, what does this translatability of all three kinds of representation 
to digital units mean with regard to the question of the content provided 
by visual representations and its usability in scientific epistemic discourse? 
Firstly, it shows that, at least in a trivial sense, there is information contained 
in all representational means that can be transferred from one medium to 
another, namely information about the respective medium itself. However, it 
is the representational relation involved that is of interest here, namely the 
question of whether what can be transferred in such translation processes can 
go beyond this trivial kind of information. That this is at least sometimes the 
case becomes clear when regarding the process of digitalisation in reverse, as 
Zachary C. Irving does. He states that: 


[c]Jomputer simulations and other computerized instruments are 
interesting case studies because they output data sets that can, in prin- 
ciple, be represented in many different styles (e.g. numerically or visu- 
ally). Since each of these stylistically distinct outputs represents the same 
data set, they should have the same content and thus, prima facie, be 
intersubstitutable in whatever scientific argument they are used without a 
change in soundness. 

(Irving 2011, 775) 


Does this mean that these different representational variants of the basic con- 
tent can then be used interchangeably in communicative contexts? 

Such an assumption seems to be immediately blocked when human beings 
are considered an essential part of the epistemic process under discussion. 
This is also the objection that Irving raises. Although it might be possible to 
transfer all kinds of information into digital units, it would be senseless to 
present these sequences of bits and bytes in scientific arguments because we 
would simply not be able to understand them.” 

Moreover, in section 3.2 of this book, it was pointed out that not all 
kinds of visual representation are equally suitable for diverse communicative 
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purposes. The background knowledge of the audience, the goal and the con- 
text of communication were highlighted as conditions essential to influencing 
the decision on how best to present the information. A similar case can be 
made with respect to the applicability of digital units. They might be useful in 
the context of information technology, but not as a basis for human epistemic 
processes. As such they have to be (re-)converted into a format cognitively 
accessible to human investigators. 

The usefulness of digital information processing in epistemic practices 
is indubitable, but digital information itself should not be regarded as an 
epistemically more basic category of representing information. It is only a 
tool, a technology that works very well, the design of which, however, could 
also have been totally different. Nonetheless, we can infer the positive result 
that the possibility of digitalising images (as well as texts) shows that they 
contain information which can be extracted and decoded by using other rep- 
resentational means. Yet it might still be objected that the kind of information 
disclosed in its processing by digital devices is more or less trivial in nature. 
For example, it does not allow us to differentiate between images that are only 
used for decorative purposes and ones that are essential to the scientific argu- 
ment. Thus, the focus of the question should be shifted slightly: are visual 
representations capable of bearing meaningful information instead of infor- 
mation in general (whereby ‘meaningful’ here refers to information relevant to 
the cogency of the arguments presented)? An instance of meaningfulness that 
plays a major role in this context regarding scientific images is the presenta- 
tion of measurement data. As our example of the Higgs image (see Figure 2.4) 
has made clear, in many instances these measurement processes are connected 
to information-technology devices. This means that the results obtained can 
be presented by more or less all kinds of representational means. However, a 
translation between different modes of representation is also possible if no 
digital information is available. In many instances, there is, for example, the 
choice of presenting correlations between numerical data values in a table or 
visualised in a diagram. Accordingly, such visual representations can indeed 
be bearers of essential information, which was one presupposition for their 
possible role in scientific discourse. 

Apart from this basic translatability, is there a kind of epistemic hierarchy 
involved when choosing between the different modes of data presentation? 
There are two points to be mentioned as a reply. Firstly, in section 2.1 of 
this book, it has already been shown that, depending on the context, some 
ways of presenting the data are more suitable to the task at hand than others. 
Secondly, such a task-orientated approach allows for the possibility that in 
some instances visual representations are epistemically superior to other 
kinds of representational means because they allow for an evaluation of 
data that otherwise would not be possible. Jutta Schickore offers an example 
in her discussion of a case study in astronomy (see Schickore 1999). Here, 
diagrams of a spectrographic analysis were used for the purpose of clarifying 
whether the data received showed two different interstellar objects or just 
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one object, doubled by a gravitational lens (see ibid, ch. 3). Schickore points 
out that, in the end, it was a matter of a qualitative visual evaluation of the 
diagrams that guided the scientists in their decision. The resemblance between 
the two diagrams supported their judgement that only one object had been 
observed and not two different ones (see ibid., 282). It can be assumed that 
this judgement would not have been possible if the scientists had only the 
numerical data at their disposal. 

In any case, it is not just resemblance and other relations between data that 
visual representations may easily reveal, but measurement errors can also be 
detected in this way. Sometimes the results have first to be depicted in a visual 
form before it can be decided what the real output is and what is just back- 
ground noise. The Higgs image (see Figure 2.4) is a good example. One might 
argue that if inconsistencies or anomalies were detected, scientists would turn 
to the numerical database for clarification. This assumed procedure is then 
taken as supporting the claim that numerical data are epistemically more 
basic than their visual presentation. However, there are two points which call 
this reasoning into question. Firstly, scientists detect anomalies because they 
evaluate those very images and base their judgements on their own visual 
experiences with those representational means. That is, detecting anomalies 
would not be possible without visual representations. Secondly, not only does 
the epistemic process start with an image, but it also ends with one: the detec- 
tion of an anomaly will surely result in the rendering of another image that 
can be used for comparative tasks. Thus, the whole process refers fundamen- 
tally to visual means. Of course, this line of reasoning should not be mistaken 
for claiming that processes of error detection and correction always work this 
way. However, it would be equally wrong to state that visual representations 
can in principle play only epistemically inferior roles. 

Summarising the results so far, the analysis shows that numerical measure- 
ment data can be translated relatively easily into visual representations and 
vice versa. In addition it has been pointed out that there are particular tasks 
and contexts that make the use of certain representational means more rea- 
sonable than others. Consequently, there seems to be no fundamental hier- 
archy in place with respect to epistemic processes when comparing these two 
forms of information presentation. Despite this initial comparability between 
both means of representation, there is another way that might provoke claims 
of epistemic difference here, namely their usefulness in the context of scien- 
tific arguments. 

In the previous section, it was highlighted that there are certain 
requirements that constitutive parts of arguments have to fulfil. They have 
to be bearers of truth values and they have to allow for certain entailment 
relations that make deductive reasoning possible. These characteristics are 
usually attributed to propositions. Moreover, it was stated that, despite there 
being rival conceptions of what is meant by the term ‘proposition’, one pre- 
dominantly uncontested characteristic is that they are supposed to be linguis- 
tically expressible. Georges Roque gives the following synopsis of the dispute 
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about visual representations in (scientific) argumentation that follows from 
accepting this condition of propositionality. “One underlying assumption 
is that arguments should be propositional. Consequently, if images are not 
propositional, therefore visual arguments are impossible unless the recipient 
extracts and builds propositions from them, which is a problematic process” 
(Roque 2015, 180). 

Of course, the degree of the difficulty involved in translating the content 
of visual representations in a propositional way is greatly dependent on how 
we conceive of the concept of propositions itself. A more moderate stance 
towards this concept can be used to more or less circumvent the whole 
quarrel. J. Anthony Blair, for example, an early defender of the thesis that 
visual arguments are possible, chooses such a strategy. He claims that there 
is no particular problem related to translating visual content into propos- 
itions, as the latter can be expressed in a variety of different ways. He states 
that “[p]ropositions can be expressed in any number of ways, including 
silence [...], but also by signs or signals [...], or by facial or other body- 
language expressions [...]” (Blair 1996, 26). As a consequence of this mod- 
erate conception of propositions, he infers that “[t]he visual expression of 
propositions, then, is familiar and relatively unproblematic” (ibid.). Whereas 
Roque argues that visual arguments are impossible as images are not prop- 
ositional, Blair shifts the perspective by claiming that propositions can be 
expressed by very different means. That is, a claim about characteristics or 
essence is turned into a question about capacities of expressibility. This shift 
in perspective then makes it relatively easy for Blair to claim that “[...] visual 
arguments are not distinct in essence from verbal arguments. The argument 
is always a propositional entity, merely expressed differently in the two 
cases” (ibid., 38). 

Roque, on the other hand, emphasises the relation between propositions 
and linguistic expressions, particularly when denying that images have the 
capacity to bear propositions. Thus, by mentioning the process of extracting 
and creating propositions, he refers to the question of the translatability of 
visual representations into linguistic expressions. This clearly more difficult 
approach to the topic can be obviated, however, by simply adopting Blair’s 
suggestion that propositions are in principle expressible in a diversity of ways, 
which would then render the task of translation redundant. 

Perini’s approach attempts to deal with the persistent problem of the 
capacity to bear truth values. This becomes clear when Blair’s later works 
(see Blair 2004) are examined, for it is here that the objection based on the 
truth-bearing capacity of propositions is taken up in earnest. He offers two 
rejoinders to the objection that, as images cannot be bearers of truth values, 
they cannot express propositions (see ibid., 47). His first rejoinder is simply 
to offer an example of what he considers to be a paradigmatic case of a 
visual argument (see ibid., 47f.), namely a political, pre-World War II cartoon 
showing an Englishman sitting beneath a pile of boulders. These boulders are 
labelled with the names of European countries. 
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The bottom boulder, sticking out but wedged under and holding up the 
rest, is marked ‘Czecho’. [...] A thick rope is attached to the out-thrust 
end of the ‘Czecho’ boulder and pulled up overhead and out of sight. 
Clearly a strong pull on that rope would dislodge the “Czecho’ boulder, 
causing the rest to come crashing down on the Englishman below. The 
cartoon’s caption reads, ‘What’s Czechoslovakia to me anyway?’ 

(Blair 2004, 48) 


The message, i.e. the proposition expressed by this cartoon, is that the pol- 
itical fate of Czechoslovakia will indeed have a major impact on England. 
Moreover, Blair points out that this proposition was either true or false at the 
time of its publication (see ibid.). 

His second rejoinder consists in highlighting that arguments are not only 
used to change the recipient’s beliefs, but also her attitudes, intentions, or 
behaviour (see ibid., 48). These latter instances do not have truth values, as he 
points out (see ibid.). Apparently, this concession implies a softening of Blair’s 
initial thesis. Yet it might be objected that, although Blair is right in claiming 
that “attitudes, intentions and conduct do not have truth value”, this does 
not imply that argumentative attempts to change them do not rely on truth- 
bearing propositions. Blair’s intention to ascribe a different important role 
to images in the context of argumentation by drawing on the psychological 
effects of visual means instead of their assumed epistemic contributions 
does not therefore undermine the thesis that they express a propositionally 
structured content. 

Be that as it may, Blair concludes by pointing to the more moderate 
achievement that he thinks he has obtained, namely that his rejoinders to this 
objection “[...] at least [...] shift the burden of proof” (ibid., 49). Alternatives 
have now to be contemplated, if his rejoinders do not convince, that create a 
way to keep visual representations nonetheless involved in the argument. 

This brings us back to Roque’s suggestion that although images cannot 
represent propositions themselves, their content can perhaps be translated 
into representational means capable of fulfilling this task. This comes down 
to questioning whether the content presented visually can be paraphrased, 
i.e. translated into linguistic expressions. Perhaps a mental reconstruction 
of arguments that is performed has to be taken into account of which the 
contemplating scholar is more or less unaware. But is the cognitive content 
of visual representations translatable into linguistic expressions in the way 
required for scientific arguments? 

Blair thinks that, in principle, such a translation is possible (see Blair 1996, 
25). Actually, he argues that visual representations pose no particular problem 
of translation. This optimistic attitude is a result of his more moderate con- 
ception of propositions. Yet he also seems to implicitly admit that there are 
limitations to the scope of such translations when he says that “[w]hether 
such descriptions or translations can be complete or fully adequate is a sep- 
arate question” (ibid., 25). The difficulty has already been mentioned that, 
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if visual representations are to be employed in communicative acts, certain 
limitations of expressibility have to be acknowledged. A related lack of image 
precision is highlighted in Elhott Sober’s account. Similar to Blair, Sober 
starts in an optimistic way by expressing his conviction “[...] that every rep- 
resentational picture” has a sentential counterpart of a certain form which 
captures the content of the picture with which it is identified” (Sober 1976, 
111). Consequently, Sober defends the thesis that a translation of visual 
representations is indeed possible. 

Interestingly, his conception of how to translate visually presented infor- 
mation into linguistic expressions also allows him to explain in what sense 
images can be called true. 


A representational picture gives it to be understood that a certain state of 
affairs obtains. If a picture is true, then the state of affairs does obtain. 
For this reason, we will identify each representational picture with an 
existential hypothesis which posits the existence of certain specified 
objects. A picture will thus be true if and only if the world contains the 
kinds of objects demanded by the sentential counterpart. True pictures 
are thus ones that have verifying instances. 

(Sober 1976, 111f., his italics) 


Sober’s suggestion is much more precise than the proposal made by Tim 
Crane, who discusses the question of whether images can have a propositional 
content (see Crane 2009). He assumes that “[flor any picture P, there is a sen- 
tence which gives the content of P” (ibid., 460). At first glance, this seems to 
be quite similar to Sober’s approach. However, Crane thinks of the senten- 
tial counterpart of images as mere descriptions of their contents, not as an 
“existential hypothesis which posits the existence of certain specified objects” 
(Sober 1976, 112), as Sober does. Crane uses this line of reasoning to con- 
clude that there is still an important difference between the image’s content 
and its description (see Crane 2009, 460). He infers from this that such sen- 
tential counterparts to the content of pictures do not support the thesis that 
images have propositional contents (see ibid., 461). Obviously, this reasoning 
is a consequence of Crane’s identification of “a sentence which gives the con- 
tent of P” with “a description of P” which allows the latter to be rather loosely 
connected to the visually encoded information. Here, Sober’s thesis is much 
more precise in pointing out that the sentential counterpart has to entail an 
existential hypothesis of a certain kind. From my point of view, this precision 
is ample reason for preferring Sober’s account to Crane’s approach. 

Yet this preference does not imply that Sober’s approach presents no diffi- 
culties. Whereas the idea that there is a kind of test to check whether images 
that are supposed to represent certain entities of the world fulfil their task suc- 
cessfully seems to be more or less uncontroversial, the moot point in this con- 
ception is, of course, Sober’s claim that it is the picture and not the correlated 
sentence that is true. Suggesting such a way of evaluating the content of 
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visual representations apparently calls for precision when it comes to the task 
of translating its content. If we want to check whether a particular state of 
affairs that a certain picture shows actually obtained, as Sober puts it, we not 
only need a vague description of what is going on in the picture, as Crane e.g. 
suggests, but a precise translation. And although Sober’s account is preferable 
to Crane’s, as the former makes clear what kind of description is called for, 
for two reasons Sober’s approach unfortunately does not fulfil requirements 
either. Firstly, there is Sober’s own discussion of the limits to what is visually 
expressible. Secondly, there is his concession that analogue pictures cannot be 
completely translated in the way suggested above (see Sober 1976, 141). 

Concerning the first aspect, Sober points out that visual representations 
lack the capacity to express negations% and disjunctions. With regard to the 
former, he notes that: 


[...] the absence of negation has two parts: On the one hand there is no 
operation on a picture that produces a picture whose interpretation is 
the negation of that of the one operated upon. On the other, predicates 
occu[r]ring in the interpretation of any representational picture are such 
that their negations never so occur. 

(Sober 1976, 128) 


Adhering to his idea of translatability, he points out that there is no logical 
way to produce two images whose sentential counterparts are contradictory, 
and consequently he claims that there is no such thing as pictorial negation. 

One might object that there are, for example, drawings in field guides for 
biologists that show a particular characteristic that dragonflies, say, belonging 
to the species Emperor always show a black mid-line stripe on the abdomen. 
Of course it is possible that this characteristic is omitted in another drawing. 
But although in such a case there is a picture showing p (black stripe), and 
the other one does not not show p, i.e. an example of an Emperor without a 
black stripe, this is not a negation of the former image but simply an image 
of a different species. 

Sober presents a similar argument concerning the possibility of disjunctive 
claims via visual representations. “There is no operation on pairs of pictures 
which effects their disjunction, and if two predicates occur in an interpret- 
ation, their disjunction never so occurs” (ibid.). No doubt, there is a broad 
variety of different pictures, but how could a relation between two of them be 
produced to express an alternative to what is shown? 

Although Sober has highlighted serious shortcomings of visual 
representations as means of scientific communication, he does not regard 
this as proving the impossibility of translations between images and verbal 
expressions. On the one hand, he does not defend the thesis that a presumed 
translatability works both ways — from the visual to the linguistic and vice 
versa. He only emphasises that the first route — from visual to linguistic 
representations — is always possible in the case of representational images. 
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On the other, he circumvents the problem by diminishing the relevance of 
those logical relations, discussed above, in human sense perception: human 
vision is developed by evolution to allow for a well-informed orientation in 
our natural environment. “For this reason, it [human perception, N.M.] is 
supposed to pick out properties of the environment that are significant in 
terms of prediction and explanation” (ibid., 131). Sober thinks that it is only 
natural properties that can fulfil the relevant task. Consequently, he claims 
that human vision is adapted to perceiving predominantly natural properties. 
He transfers this line of reasoning via an analogy to picture perception. By 
claiming that the basic cognitive processes in both instances are quite similar, 
he takes it that “[h]ence one should expect pictorial interpretations to fail 
to include irreducible disjunctions and negations, since these fail to pick out 
properties” (ibid., 131). What Sober apparently hints at is that, as we decode 
images via perception, the presentation of visual information has to obey the 
same limitations (or environmental accommodations, to put it in a positive 
way) as everyday perception does. In short, if we do not observe negations in 
the world, it should come as no surprise that they do not belong to the reper- 
toire of what is expressible via visual representations. 

Of course, this is quite a controversial claim that Sober inserts into his 
argumentation at this point. Although he might be right that human sense 
perception is particularly adapted to noticing certain entities in the world 
better than others, this does not constitute a reason that the expressibility of 
pictorial representation should be limited. Why should images not be able to 
present information that we usually do not perceive in our natural environ- 
ment? Why should it not be possible to learn with this respect? Many scientific 
visualisations probably exhibit phenomena unfamiliar to everyday percep- 
tion: the Higgs image (see Figure 2.4), for example. 

Anyway, what should have become clear is that there are certain rela- 
tional properties, such as negations, that cannot be expressed via visual 
representations.” Now, for the initial concern about questions of translata- 
bility from one representational mode into another, this means that there are 
certain constraints on transferring information from the linguistic to the visual 
domain. Although this does not pose a problem for Sober’s own thesis that 
all representational images can be translated into linguistic expressions, a dif- 
ficulty arises that does affect his project, namely translations from the visual 
to the linguistic domain. Here, complications arise when we want to translate 
what he calls analogue pictures. 

What are analogue pictures in Sober’s sense? 


In saying that a representational system is analog, we are claiming that 
there is a property of representations which the system treats as sig- 
nificant, and which is such that if there were a continuum of values 
of the property, there would be a continuum of significantly different 
representations. 

(Sober 1976, 139) 
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Because of these possible continua of property values, an analogue system 
of representation is usually richer than a digital one regarding the informa- 
tion it presents. Although translation into a digital system — such as natural 
languages — is possible, it implies a loss of information and, thus, “a loss in 
precision”, as Sober points out (ibid., 140). A translation of the content of 
analogue pictures into linguistic expressions can therefore be achieved only 
approximately, that is, with a considerable lack of precision (see ibid., 141). 
This lack of precision is the result of there being a number of very different 
possible sentences derivable from an analogue picture via translation, which 
also limits Sober’s initial thesis that the truth of pictures can be proven via 
verifying instances expressed in the sentential counterparts of images. If there 
is a variety of different sentential counterparts, no definite verifying instances 
can be stated. 

To sum this up, Sober’s account of the relation between linguistically and 
visually presented information seems to be correct. We are obviously able 
to describe what we see in representational images to a certain degree, and 
can thus take this as supporting the thesis that there is a cognitive content 
of visual representations that can be used to transmit certain information. 
Moreover, the discussion of Sober’s ideas also makes plain that there are cer- 
tain limitations to what is expressible in the visual format, but also highlights 
the fact that these limitations hold vice versa, i.e. that the informativeness of 
the visual content can outstrip the capacity to represent information in lin- 
guistic expressions. As a result, a reduction of visual information to the latter 
is not possible. Or, as Sober puts it: 


[h]ence, not all pictorial systems are reducible to impoverished linguistic 
systems of a certain kind. The relation of analog pictorial systems to 
language is more complicated: With respect to logical operations on 
representations, linguistic systems are more powerful, but with respect to 
expressing specifically visual relations between posited objects, linguistic 
systems can be more impoverished. 

(Sober 1976, 139) 


It might be objected that images not only contain more information, but 
also raise the issue of ambiguity by this. Hence their richness of information 
might turn out as a particular shortcoming, especially in the context of sci- 
entific arguments. The soundness and validity of an argument depend on the 
precision of its premises and its conclusion. If visual representations cannot 
guarantee this, they might be inappropriate as representational means in this 
context. 

A rejoinder to this objection has been worked out by Blair. He highlights sev- 
eral aspects diminishing the sceptical attitude towards visual representations 
in scientific arguments (see Blair 2004, 46f.). Blair states that ambiguity and 
vagueness are not attributes exclusively attached to the visual domain, but 
also to linguistic expressions. Nonetheless, difficulties of understanding are 
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mostly ruled out by information provided by the context of uttering an argu- 
ment. Moreover, visual arguments are usually mixtures of verbal and visual 
representations so that the former can (and are) used to disambiguate the 
latter. 

This fits neatly with what was stated earlier about taking the whole com- 
municative act into account when considering the meaning of particular 
visual representations. It is advisable here to follow Kjorup’s account of the 
topic. Similar to Blair’s idea, Kjorup points out that the context and the com- 
bination of visual and linguistic expressions will enhance our understanding 
of visually transmitted information, as the former will help to disambiguate 
the latter. Thus, we can conclude that ambiguity and vagueness pose no par- 
ticular problems with respect to visual arguments. 

What would be more interesting to know is whether the comprehensive 
informativeness of visual representations, pointed out by Sober and others, 
also entails information that in principle cannot be transmitted by other repre- 
sentational means. If this were the case, not only would the quantity of infor- 
mation of certain images be different from other representational means, but 
also their quality, i.e. the kind of information that can be transmitted via 
visual representations, would differ. 

Taking a look at concrete examples suggests that this might indeed be the 
case. For instance, Perini discusses the question of whether a micrograph can 
be translated into linguistic expressions. Pointing to the same continuum of 
property values that Sober highlighted above, she comes to the conclusion 
that a reduction to verbal language is not possible. She offers the following 
reason as a rationale for her inference: 


[b]ecause the form of pictorial symbols is correlated exactly with features 
of their referents, a dense range of extremely complicated properties can 
be expressed by a pictorial system. This kind of system is extremely useful 
in science because it allows for the representation of properties, whether 
or not the vocabulary exists to refer to those properties with linguistic 
representations. As a result, a verbal description of the shape of the figure 
will convey less specific information about the shape of the sample than 
the figure itself. 

(Perini 2005c, 923) 


Here, she highlights the fact that visual representations can transmit infor- 
mation that have no linguistic counterpart, i.e. no linguistic expressions avail- 
able to adequately describe, let alone translate, what is occurring in the image. 
It is this characteristic feature of visual representations, namely that they 
enable communication about entities and states of affairs without the appro- 
priate words available for them, which makes them particularly useful in sci- 
entific arguments (see Perini 2010, 144). 

However, if we take it that propositions are connected to linguistic 
expressions, at least in the sense of being expressible by them, then Perini’s 
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example suggests the interesting conclusion that some scientific images can 
transmit non-propositional content. Moreover, if images contain such a kind 
of cognitive content, then the strategy of translating their content in order to 
adhere to the traditional conception of proposition-based arguments will be 
blocked. It is also this line of reasoning that motivates Blair’s claim that “[flor 
visual argument to represent a radically different kind of argument, it would 
have to be non-propositional” (Blair 1996, 34). 

Moreover, as Perini’s example makes clear, it would be wrong to dismiss 
visual representations as devoid of substantial information. On the contrary, 
they can contribute a kind of information that apparently cannot be trans- 
ferred by linguistic means. Yet how is this information perceived, how is it 
cognitively processed so that it can play a role even in scientific arguments, 
and can knowledge be acquired in this way if the latter is usually considered 
to be propositional in kind? 

Obviously the main process of cognitively accessing the content of visual 
representations is by visual perception. Thus, the next question to answer 
must therefore concern how perception is connected to propositional content 
or to non-propositional content respectively. Analysing these relations will 
offer an explanation of the possibility that the non-propositional content of 
images can play a role in scientific arguments. 


4.2.2 Perception and non-propositional content 


This section will be concerned with the nature of the content of perception. In 
particular, the question will be discussed whether this content is propositionally 
structured. The answer to this question will facilitate greater precision about the 
kind of content perceivable when regarding scientific images. Two aspects of 
the following line of reasoning are to be highlighted at this initial stage: firstly, 
it is particularly the theoretical assumptions put forward by Christopher 
Peacocke that will afford a connection between the debates in the philosophy 
of perception and in picture theory. Secondly, although the main focus of the 
analysis is on perception as an epistemic source, it will turn out that the inter- 
play between different sources of knowledge is of relevant consideration in 
order to explain sufficiently what kinds of content can be transmitted in image 
perception as well as in perception in general. 

Philosophical considerations of perception are related to different branches 
of this academic discipline. There are phenomenological approaches as well 
as epistemological ones, but the topic is also connected to questions of the 
philosophy of mind.” Regarding the current topic, two constraints need 
mentioning that will restrict the set of theoretical approaches that have to be 
taken into account: epistemological questions on the one hand and the cog- 
nitive mechanisms of vision on the other — although other sense modalities 
might play a role as well. In the following, only those parts of the debate about 
perceptual knowledge that are closely related to epistemological questions of 
picture perception in science will be examined. 
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It is not usually called into question that knowledge ascriptions are 
made, and are also often justifiably allowed to be made, based on the epi- 
stemic source of perception. It seems to be perfectly admissible to reply 
“Because I saw that p” to the question “How do you know that p?”. This 
becomes particularly apparent when considering the debate about know- 
ledge by testimony. Here, perception is commonly regarded as a more or less 
uncontroversial exemplar of an epistemic source, whereas philosophers still 
dispute whether testimony can have parity with perception in this respect. 
Participants of this debate seemingly have no qualms about regarding per- 
ception as an epistemic source. What is meant, then, when sources of know- 
ledge are discussed?” 

Robert Audi specifies the epistemic achievements of those sources in the 
following way: 


[a]s I am understanding sources of knowledge, and as they are generally 
conceived in philosophical literature, they are not just where knowledge 
comes from; they also provide the knower with grounds of knowledge. 
Grounds are what it is in virtue of which (roughly, on the basis of which) 
one knows or justifiedly believes. [...] sources indicate the kinds of 
grounds to expect a person to have when a person has knowledge through 
that source. 

(Audi 2002, 82, his italics) 


Thus, epistemic sources not only tell us how certain beliefs are acquired, 
i.e. which cognitive resources are used in in this process, but also yield reasons 
that can justify the corresponding beliefs. Mentioning an epistemic source as 
a reply to the question “How do you know that p?” is commonly regarded 
as a prima facie justification of the corresponding belief. Hence, from an 
internalist’s point of view, epistemic sources differ from mere causes of beliefs 
in that they provide us with reasons to think that the belief acquired is true, 
as Thomas Grundmann points out (see Grundmann 2008, 453). From an 
externalist’s point of view, epistemic sources specify reliable ways (or methods) 
of belief formation. Of course, these cognitive processes are not infallible, but 
philosophers acknowledge them as usually leading to true beliefs. They also 
agree that epistemic sources can yield knowledge, i.e. propositional knowledge. 
Grundmann makes clear that calling x an epistemic source implies a positive 
epistemic evaluation of x. Making use of x in one’s cognitive processes will 
usually imply that one will gain knowledge from x (see ibid., 455). 

Perception is usually regarded as belonging to this set of epistemically 
distinguished sources. In the debate about knowledge by testimony, perception 
is even regarded as a paradigmatic example of such a source. Acknowledging 
this means accepting that perception enables epistemic subjects to gain prop- 
ositional knowledge.*° 

Despite the agreement of many philosophers on this positive epistemic 
assessment of perception, and also their acknowledgement of the thesis 
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that it is propositional knowledge that is at stake when contemplating the 
achievements of epistemic sources in epistemology, there is no consensus 
on the nature of perceptual content (see e.g. Siegel 2015). A primary con- 
sideration here is whether this content is propositional or non-propositional 
in kind. A propositional content would provide accuracy conditions with 
respect to claims concerning what has been perceived by a particular speaker 
(see ibid., sect. 2). As was pointed out above, considered in the Fregean trad- 
ition, a proposition is commonly regarded as an abstract object which is 
either true or false. Scholars also agree that propositions can be expressed by 
linguistic statements. Beyond this, however, there are also concepts that are 
related to both propositions and to language.*! Now, some think of concepts 
as the “constituents of propositions” (Margolis and Laurence 2014, sect. 
1.3). Regarded this way, the question of whether the contents of perceptions 
are propositional or non-propositional in kind also implies the question of 
whether their contents are conceptual or non-conceptual. Saying that those 
contents are propositional would then also mean to claim that they are con- 
ceptual, and, following Frege, these contents can then be expressed by verbal 
statements. 

However, the precise relation between concepts and propositions is far 
from clear and the discussion about the contents of perception therefore 
becomes even more complex when taking this relational aspect into account. 
Grundmann, for example, questions the cognitive process involved in percep- 
tual activities in this respect. He wonders how perceptual content is turned 
into propositional knowledge if perception transmits non-conceptual 
contents, an assumption which he defends (see Grundmann 2008, ch. 7.1.2). 
As a rationale for his thesis that the content of perception is non-conceptual 
in kind, Grundmann states, amongst other things, that we are able to perceive 
entities, for instance, different hues of a colour, for which we have no concepts 
(see ibid., 492).3? 

In particular, philosophers ask how sense experience and perceptual beliefs 
are related. Perceptual beliefs are regarded as propositional attitudes here. 
From a Fregean point of view, fulfilling their task as accuracy conditions 
also implies being expressible by verbal statements, which seems to presup- 
pose that those contents are conceptualised. Grundmann makes clear that 
we should not simply identify perceptual content and perceptual beliefs 
(see ibid., 488f.). In particular, cases of optical illusions show that sense 
perceptions are quite robust even when challenged by background know- 
ledge. That is, although the epistemic subject is aware of being misled by her 
perceptual apparatus, because she knows how the optical illusion at hand 
works, she is not able to change her perceptual experience. Consequently, 
she will have a particular visual impression, but will not believe what she sees 
because she knows about the illusionistic mechanism at work. Interrupting 
the process of belief formation in such a way, however, seems to be rather 
exceptional because we do not usually question what our senses tell us about 
the world but just believe what we perceive. More often than not, perceptual 
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experiences are then turned into perceptual beliefs. For the moment, the 
answer to Grundmann’s question about how exactly this process of trans- 
formation works will be left open. It will be explained below when we con- 
sider Peacocke’s theory of non-conceptual perceptual content. What is of 
importance to the current investigation, however, is noticing the fact that 
this process usually works reliably. That is, perceptual contents are turned 
into perceptual beliefs, even though the former can be non-conceptual in 
kind, whereas the latter are commonly regarded as propositional attitudes 
that presuppose a conceptualised content. 

In the context of science, Hentschel offers an illustrating example of 
observations that yield non-conceptual content and of scientists’ subsequent 
attempts to conceptualise what they have observed. He describes the first 
observations of the surface of the sun in the nineteenth century (see Hentschel 
2000, 23ff.). The solar structures, now known as granulation, visible with con- 
temporary telescopes, were completely unknown to astronomers of that time. 
Thus, they not only lacked the words to describe what they had observed, 
but also had no conceptual background knowledge about the mechanism 
producing this phenomenon. As a strategy to circumvent the descriptive 
problem, astronomers made use of different metaphors, e.g. “willow leaves in 
an ocean of fire”, “rice grains”, etc. (ibid., 23) to report their observational 
results. Moreover, these metaphors were also used to pictorially illustrate 
their observations in their publications. Here, Hentschel points out that those 
images were particularly important to subsequent research activities about 
the related phenomenon, as they influenced significantly the way scientists 
observed it later on: 


[w]hat is at issue here is far more than terminology: Behind these com- 
peting ‘descriptive labels’ are mutually exclusive options on how best 
to see a new feature, that is, which Gestalt is assimilated to their visual 
impressions. [...] And these published illustrations in turn strongly 
influenced the perception of other observers [...] about what they 
anticipated seeing, that is, recognizing, in their telescopes. 

(Hentschel 2000, 23ff.) 


By making use of metaphors (linguistically and visually), those scientists 
started conceptualising the phenomenon of granulation that they were 
observing. Those metaphors allowed them to refine their observations by 
offering a point of comparison to already-known phenomena, to exchange 
ideas about their observational results and, later, to develop hypotheses to 
explain the mechanism in the background. This example makes plain that 
human observers are apparently able to receive perceptual information, 
although they lack the concepts to explain and to describe in concrete terms 
what they have seen. 

Here we find an explanation for why it is possible that some scientific 
Images can transmit information we cannot describe by using linguistic 
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expressions as we do not have the relevant words to do so — i.e. the thesis 
defended by Perini above. On the one hand, images can be used to transmit 
information about phenomena during the process of their conceptualisation, 
as Hentschel’s example demonstrates, and, on the other, scientific images — 
in particular those working in a causal way, i.e. that are causally related to 
the object under investigation — can also contain non-conceptual information 
about the entity they represent; and, as it is through perception that we grasp 
such non-conceptual information, so human observers can learn about those 
entities by regarding the respective images. 

Philosophical analyses of perceptual content also allow us to learn what 
non-conceptual contents might be like. Postulating such a kind of content is 
the result of acknowledging the above-mentioned thesis that the content of 
perceptual experiences and corresponding beliefs are not identical. Susanna 
Siegel calls a corresponding thesis of identity which explicitly refers to concepts 
in this context “experience conceptualism” and defines it as follows: “[flor any 
object x and any property F, a subject has an experience as of x being F only 
if she has concepts of x and F, and deploys those concepts in the experience” 
(Siegel 2015, sect. 6). The relation between beliefs and experiences expressed 
in this statement can be grasped more quickly by identifying the contents 
of beliefs with the contents of experience. Siegel calls this the “same-con- 
tent” thesis: “[flor any experience as of an object x having a property F, if the 
experience has content p, then it is possible to have a belief with content p” 
(ibid). 

Both of these theses are challenged by the fact that perceptual experiences 
can apparently be informatively richer than the concepts people are able 
to deploy, as Siegel points out (see ibid., sect. 6.1). Consequently, it can be 
argued that if non-conceptual information is acquired via perception, then (at 
least) some perceptual contents are not bound to concepts in the first place. 
The example of the first observations of the granulation of the solar sur- 
face illustrates this point nicely. Scientists at that time had no concept of the 
phenomenon at their disposal. That is, they could neither verbally describe 
what they had observed nor explain what was occurring, namely what caused 
the phenomenon they were observing, what mechanisms were responsible for 
causing it, nor why it looked the way it did. Nonetheless, they were able to see 
those particular structural aspects of the solar surface. 

A first approximation to the concept of ‘non-conceptual content’, then, 
consists in acknowledging the fact that we are dealing with a contrastive 
concept here, as José Bermudez and Arnon Cahen point out (see Bermudez 
and Cahen 2015, sect. 2). That is, our understanding of the term ‘non- 
conceptual content’ depends crucially on how we conceive of the term 
‘conceptual content’. Consequently, there is a variety of hypotheses avail- 
able about what exactly the non-conceptual content of perception might 
be like. 

The contrastive nature of the term ‘non-conceptual content’ is also 
highlighted by Christopher Peacocke (see Peacocke 2001a, 243). He briefly 
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addresses the relation assumed between concepts and language in his discus- 
sion of this topic. 


If someone holds that a concept user must have a language in which he can 
express at least some of his concepts, that is a substantive, nondefinitional 
thesis that needs to be established. [...] It should, however, be uncontro- 
versial that any content that can be expressed in language by the use of 
an indicative sentence, including sentences containing indexicals and 
demonstratives, will be a conceptual content. 

(Peacocke 2001a, 243) 


Thus, having concepts at one’s disposal does not necessarily urge us to admit 
that we possess a language to express those concepts. Consequently, the 
inability to speak, either temporarily or permanently, is no necessary indi- 
cation that the corresponding person does not have concepts at her disposal. 
It is only the weaker thesis, namely that what has already been formulated 
in language is automatically related to concepts, that Peacocke holds to be 
uncontroversial when considering the question in what sense concepts and 
language are related to each other. Thus, Peacocke’s ideas caution against 
claiming too close a relation between language and concepts. 

Can the nature of non-conceptual content of perception be described in a 
more substantial way? Here, Peacocke’s approach is particularly interesting, 
as his considerations bridge the gap between the discussions in the philosophy 
of perception and in picture theory. With respect to the latter, he thinks that 
philosophers should take into account results from cognitive science, in par- 
ticular from theories of perception (see Peacocke 1987, 383, 404). In the 
following summary, together with a discussion of some related examples, 
a recapitulation of what he suggests in the context of his theory of depic- 
tion will help to clarify why the transmission of non-conceptual content is 
important in epistemological terms and what kinds of cognitive processes are 
at work. 

As was explained in section 2.2.1 of this book, Peacocke uses the concept 
of the “visual field”, borrowed from theories of perceptual experience, to elu- 
cidate where exactly to look for resemblance relations between an object and 
its depiction. According to this assumption, it is not the direct properties of 
an object and its depiction that are compared, but properties belonging to the 
perception of image and object. That is, what is compared are experiences 
of the object and of the image in the visual field of the perceiver. The visual 
field can be regarded as a kind of mental intermediate plain between perceiver 
and the object of perception. It is then in experiences of the image and the 
object in this visual field that resemblance relations are detected (see ibid., 
386). Now, if the perceiver possesses the relevant concept to identify and label 
the object correctly, i.e. if she makes use of this concept as a comparative 
instance to her experience of the image at hand, Peacocke suggests that what 
is presented to the perceiver in the visual field is then “F-related” (‘F’ refers 
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to ‘field’) to that very concept (see ibid., 387). The Altamira cave painting 
is a good example (see Figure 1.1). According to Peacocke, the viewer will 
experience a particular similarity in the shape of the painted animal and of a 
living example of the same species in her visual field. Moreover, if the viewer 
possesses the concept of an aurochs, she can cognitively proceed by calling the 
cave painting an image of an aurochs. 

But what happens if the corresponding concept is not available to the 
viewer? Here, Peacocke brings in the idea of a non-conceptual content in his 
theory of depiction. He argues that there are images — most of all, abstract 
paintings — that are experienced as having representational qualities,” that is, 
they will be experienced as showing particular shapes, although the viewer 
does not possess the concepts that would allow her to describe her visual 
field experience linguistically. Moreover, Peacocke thinks that the content 
of the viewer’s experiences can nonetheless be “assessed as veridical or non- 
veridical” (ibid., 395). 

To explain why it is that the viewer’s experiences of the respective image 
represent the latter as showing certain shapes, although the relevant concepts 
are not available, Peacocke brings in what he calls the “analogue content” 
of pictures (see ibid.), which he considers to be non-conceptual in kind. 
Moreover, he is convinced that assuming the possibility of such a kind of 
content being transmitted in at least some processes of picture perception is 
crucial as: 


[...] it explains how there is room in the space of logical possibilities for 
something that actually occurs: the acquisition of a recognitional cap- 
acity for a kind of object (or an individual) by seeing a depiction of a 
member of that kind (or of that individual). 

(Peacocke 1987, 395) 


An illustrative example of this is provided by images in field guides for 
biologists. Consider the following case: I observe a particular bird of prey 
circling Lake Geneva. As I do not know what this species is called, I want to 
identify it with the aid of a bird guide. Before consulting this book, all I know 
about the animal is that it is a bird of prey hunting for fish and has a certain 
appearance during its flight, that is, I know how its tail is formed, perhaps its 
colour, and I have a rough estimation of its size. So, my observation and my 
interest in the species have already triggered my attempt to conceptualise my 
observation. Nonetheless I still will not be able to classify the bird with the aid 
of a verbal description, as there are too many uncertainties correlated with 
my brief observation. The bird guide, however, provides a series of drawings 
of flying birds of prey that I can use as a means of comparison. That is, I can 
now compare these pictures with my mental image, derived from my memory, 
of what the bird looked like during its flight. Obviously, what finally makes 
a classification possible is this comparative task when considering similar- 
ities and dissimilarities between the appearance of the bird that I remember 
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and the respective image printed in the book. Hence, although resemblance 
relations might not be the non plus ultra explanation of the phenomenon of 
depiction in general, they nevertheless prove to be an important aspect in our 
handling of images. In particular, they help us to conceptualise a phenomenon 
by using images as teaching tools. What is made plain by this example is that, 
like perception in general, picture perception can transmit non-conceptual 
contents, for example about similarities and dissimilarities between an object 
observed and a drawing in a field guide. They can contribute to our acquisi- 
tion of concepts and be turned into linguistically expressible beliefs later on. 
Therefore, Peacocke’s concept of “analogue content” connects picture per- 
ception with our broader visual capabilities. 

Peacocke suggests a similar line of reasoning. He thinks that, in general, 
images can help us to recognise later on what they depict. That is, by showing 
you an image of a flying swift,*+ you will be able to discern one in the wild. 
You will know, for example, how to distinguish a flying swift from a flying 
swallow, although you have not had the relevant concepts to explain and 
describe this difference beforehand. In this sense, images can contribute to 
the conceptualisation of phenomena. It is the non-conceptual content of 
perception that explains how we are able to acquire observational concepts 
in the first place (see Peacocke 2001a, 252ff.). That we finally possess the rele- 
vant concept is indicated by our ability to correctly recognise the respective 
entity afterwards. 

This, then, is how analogue content works and why it is important. But 
what exactly does it consist of? Siegel claims that non-conceptual content 
in Peacocke’s sense can be subdivided into “scenario content” and “proto- 
propositional content” (Siegel 2015, sect. 6.2). For both of them, it is essen- 
tial to note that Peacocke distinguishes clearly between the properties of the 
content of an experience and the properties of an object, or event, that is 
perceived. These properties can show relevant similarities, but need not do so. 

An example of this phenomenon concerns distorted experiences. That is, 
perceiving an entity x from an oblique angle — or from a non-optimal point of 
view — will present x with a distorted shape in our experience. Its shape might, 
for instance, appear squeezed or stretched — although x’s actual properties of 
shape have not been altered in any way.’ 

The content of experience can thus diverge from what has been perceived 
by the viewer. There is an additional dimension, so to speak, entailed in the 
content of our experience, namely the way we perceive those entities. 


So, in characterizing the fine-grained content of experience, we need the 
notion of experience representing things or events or places or times, 
given in a certain way, as having certain properties or as standing in cer- 
tain relations, also as given in a certain way. Henceforth, I use the phrase 
the content of experience to cover not only which objects, properties, and 
relations are perceived, but also the ways in which they are perceived. 
(Peacocke 2001a, 241, his italics) 
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Figure 4.4 Non-conceptual content of perception: the same shape can be experienced 
either as a regular diamond or a square. 


Source: own image. 


To illustrate what kind of information is added when our perceptual con- 
tent contains the ways we perceive things, Peacocke discusses an example 
presented by Ernst Mach. According to Mach, Figure 4.4 can be perceived 
either as a regular diamond or as a square, without the actual properties of 
the figure being altered. Consequently, the way that we perceive the figure is 
either as a regular diamond or a square. This information is added to our 
content of experience of Figure 4.4, although the properties are not changed. 

Now, although the content of our experience and what has been perceived 
by us can diverge in the way just explained, the latter is nonetheless the cause 
of the former (see Peacocke 2001a, 248). This is also why Peacocke thinks 
that the content of our experience can indeed be assessed with respect to its 
correctness (see ibid., 240f.) which implies that we cannot just make up our 
minds. What can be perceived is restricted by this causal connection between 
the entity and our content of its experience. 

Within this framework, Peacocke highlights two different kinds of non- 
conceptual content of experience. The first one is what Siegel calls “scenario 
content”. She explains that “[s]cenario content is a set of ways filling out the 
space around the perceiver, relative to an origin and axes marking directions, 
that is consistent with the perceiver’s experience being veridical” (Siegel 2015, 
sect. 6.2). Or in Peacocke’s words: “[s]cenario content is essentially the notion 
of a spatial type, a type an experience can represent as instantiated in the 
volume of space around the subject of the experience” (Peacocke 1994, 420). 
Thus, the scenario content is related to the location of the perceiver in her 
environment during a concrete perceptual episode. Usually the scenario con- 
tent of an observer is related to her conceptual states. Yet this does not imply 
that the scenario content itself is (completely) conceptualised (see ibid., 422). 
Think, for example, of the concepts of being left or right. Apparently we are 
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able to experience a cat crossing our path from the left to the right although 
we might not possess the concepts of ‘left’ and ‘right’. We would not say that, 
before acquiring them in our childhood, we were not able to entertain the 
relevant experience. 

The second kind of non-conceptual content is called proto-propositional 
content. This type of experiential content is needed as there are instances when 
the scenario content of a perceptual experience remains the same, although 
these experiences differ phenomenally (see Siegel 2015, sect. 6.2). An example 
that Peacocke discusses with this respect is Figure 4.4. Although both the 
shape and the scenario content remain the same, this figure can be regarded 
either as a square or as a regular diamond. What changes is “the way in which 
some property of relation is given in the experience” (Peacocke 2001a, 240). 
Yet, despite this last explanation, it remains difficult to understand what kind 
of information is added by this way of givenness in the perceiver’s experience. 
Apparently it is not about the perceptual setting — i.e. directions, relations, 
lighting conditions, etc— as this is what the scenario content is supposed to 
transmit. 

Nonetheless, the ways things are given in our experience are essential, as 
Peacocke makes plain, because they provide the foundation to conceptualise 
what we have perceived. He points out that concepts presuppose some kind 
of raw material or information, so to speak, from which they can be formed. 
Thus, the non-conceptual content of our experience precedes the process 
of conceptualisation (see ibid., 245f.). Moreover, those ways of givenness in 
experience correspond to certain kinds. “On reflection, it should not be a sur- 
prise that for each way, there is a specific kind such that the way is intrinsically 
a way for something of that kind to be perceived” (ibid., 246). Concerning 
Figure 4.4, this means for example that the way this figure is given in our 
experience can correspond either to the perception of a square or a regular 
diamond, but not to the perception of a circle. Experiencing the figure as 
having four edges is the condition that allows us to subsume it in a certain 
category and this way of experiencing Figure 4.4 excludes it from belonging 
to the set of phenomena that can be called a circle. It is the way Figure 4.4 is 
given in our experience that allows us to judge that this is a square if we do 
possess the concept of squareness, i.e. if we know that a condition of being a 
square is ‘having four edges’ (see ibid., 253). This is why the way of givenness 
in our experience is proto-propositional. It is the foundation for respective 
judgements which allow us to formulate corresponding that-clauses, but it is 
not conceptualised as such. 

Peacocke explains what is going on in such instances when discussing 
how the transition from non-conceptual content to observational concepts is 
achieved. He suggests that perceivers: 


[...] will be displaying a sensitivity to a particular boundary. This is the 
boundary between those states which are, and those states which are not, 
mentioned in the possession condition for a given concept such as square. 
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[...] One, perhaps the basic, way to make this transition is to ask questions 
that are in the first instance not about concepts, but about the world. [...] 
One can answer such questions by drawing on one’s ordinary, ground- 
level abilities to react rationally to one’s perceptual states in coming to 
make judgements about the world. 

(Peacocke 2001a, 258) 


Thus, the thesis is that we do need the non-conceptual content of experience 
to acquire the concepts that allow us to make judgements about the world, to 
describe and communicate about it. 

Here, we can add an important point related to this nexus between the non- 
conceptual content of experience and the acquisition of concepts. Peacocke 
points out very clearly that it is because of this kind of content that this 
latter cognitive achievement can be accomplished. He thereby rules out an 
important rival to his theoretical approach, namely discussing states of per- 
ception instead of contents of perception in this debate. Tim Crane suggests 
that Peacocke’s theory of scenario contents can be used also as a rationale to 
defend such a state-orientated view of perceptual experience. 


Peacocke’s theory might seem to be a version of the content view, but 
properly understood, it is a version of the state view. The reason why 
a state with scenario content is non-conceptual is because S is not 
required to possess any of the concepts that canonically characterize 
the scenario in order for S’s state to be canonically characterized in 
terms of it. 

(Crane 2009, 468) 


Peacocke, however, highlights the possibility of making transitions from 
non-conceptual to conceptual contents of experience. Apparently this is what 
we experience in the processes of learning, and it seems to be much more 
plausible to assume that our concepts are formed — or carved out — of a raw 
content, namely a non-conceptual content, than that they appear because of 
their relation to a non-conceptual state. 

In the following, we will discuss some clarifying remarks that will bolster 
Peacocke’s claim concerning how to conceive of the cognitive process of con- 
cept acquisition. Two supplementing ideas referring to human developmental 
history are worth considering here. Those ideas will shed some light on what 
Peacocke might have had in mind when writing about proto-propositional 
content. 

Firstly, it can be assumed that our perceptual apparatus has been crucially 
shaped by evolutionary processes. That is, only those of our species survived 
who were able to react reasonably adequately to what they had perceived. 
Forming proto-propositional contents about one’s environment, about food, 
predators, members of and threats to one’s community apparently seem to 
have been evolutionally advantageous. Such a line of reasoning appears to 
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lie at the heart of Peacocke’s thesis that human beings are sensitive to certain 
boundaries concerning the conditions of the possession of concepts. 

Secondly, as mankind has always been a social species, it is strongly advis- 
able to take into account that the process of learning has not to be mastered 
by the individual alone. On the contrary, we can, and usually do, rely on 
members of the family, relatives and other human beings in general. That 
is, we rely on the division of epistemic labour. Thus, it is rational to assume 
that not only perception and testimony often work hand in hand as epistemic 
sources when it comes to language acquisition (see Bermudez and Cahen 
2015, sect. 4.1), but that they also co-operate when we learn about concepts 
in the broader sense. Parents show their children the entity they name for 
them (see Harris 2012, 8). By repeating this show-and-tell event and by the 
children’s listening to subsequent parental explanations about what is char- 
acteristic of those entities, children will come to be aware of the boundaries 
of certain concepts and, later, will know how to deploy them in language. 
Requiring that concepts always come first would apparently undermine the 
whole cognitive process just described. 

However, it would be equally wrong to deny that the ability to deploy 
concepts can contribute much to what we are able to observe, as Chalmers has 
highlighted (see Chalmers 1999, 10ff.). His concern is that observational facts 
have to be formulated as statements to be of use in scientific reasoning — a claim 
that reminds us strongly of the controversy about the role of images in scientific 
discourse. As a result of this assumption, Chalmers thinks that we are in need 
of an “appropriate conceptual framework and a knowledge of how to appro- 
priately apply it” (ibid., 11) to formulate those sentences. Moreover, he shows 
that the better — i.e. the more appropriate to the task at hand — our observa- 
tional concepts are, the more detailed our scientific observations can become. 

This interplay between concepts and observation also appears in 
Hentschel’s example of the first observations of the solar surface. Not only 
does he demonstrate the difficulties that those first observers had to face in 
trying to understand the visual information their telescopes conveyed to them, 
he also explains how their strategy of analogical reasoning, 1.e. the applica- 
tion of metaphors to describe their observational results, influenced later 
observations of the respective phenomenon. Hentschel describes this process 
as a search for and application of different Gestalt concepts (see Hentschel 
2000, 23). Moreover, he notes that published illustrations influenced what 
other scientists expected to see and actually observed (see ibid., 25). We have 
already become acquainted with what Hentschel describes here through 
Fleck’s ideas about scientific communication processes and the place of visual 
representations therein (see section 3.2.3 of this book). 

Fleck makes plain that often there is a feedback loop between popular 
science and expert knowledge (see Fleck 1979, 115ff.). In particular, he 
shows how images, produced as means to communicate results in a simplified 
manner to laypeople, influence the experts’ cognitive processes. “But what was 
initially meant as a means to an end acquires the role of a cognitive end. The 
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image prevails over the specific proofs and often returns to the expert in this 
new role” (ibid., 117). Thus, to use Hentschel’s terminology, once a concrete 
pictorial gestalt — putatively formed by deploying certain concepts — has been 
chosen by the expert to present her results, this gestalt will determine what she 
will henceforth observe and investigate. 

Moreover, the above discussion of the non-conceptual content of percep- 
tion allows us to explain why this initial choice of a pictorial or descriptive 
gestalt does not lead into an epistemic dead end and, thus, to a cognitive 
standstill if those choices are not adequate. Fleck describes what might have 
been expected, namely a particular tenacity of thought styles and what he 
calls a “harmony of illusion”, i.e. the tendency to explain phenomena in 
accordance with the prevalent thought style or even to ignore divergent phe- 
nomena which cannot be accommodated in the theory at hand (see ibid., 
27ff.). Consequently, the question arises how a change in thought styles is 
possible if this is necessary. How can we perceive something in accordance to 
a different gesta/t-concept if the one initially chosen does not fit? 

Although Fleck discusses in detail the effects of the harmony of illusion, 
he does not deny that persistent anomalies can surface in those systems and 
urge scientists to deal with them (see ibid., 27ff.). On the contrary, he expli- 
citly describes two different stages of scientific observation as a result of such 
difficulties. Firstly, he mentions a “vague, initial visual perception” which 
he calls “unstyled” (ibid., 92) and, secondly, the “direct perception of form 
[Gestaltsehen]” which “requires being experienced in the relevant field of 
thought” (ibid.). Thus, concrete Gestaltsehen, as explained above, is preceded 
by a weighing of the pros and cons of the various alternatives in order to 
conceptualise the phenomenon at hand. This balancing of alternatives is 
made possible by the fact that the individual is simultaneously a member of 
different thought collectives providing her with divergent suggestions of how 
best to explain the phenomenon, i.e. what concepts to choose that can provide 
an explanation expressed in words or pictures known to the scientist. 

Here, we can grasp how scientists deal with what is non-conceptually 
provided by perception to human cognition in their observational processes. 
It is this epistemic strategy, i.e. the application of concepts borrowed from 
different thought styles as explanatory means to persistent anomalies, that 
turns metaphors into the seeds of new theories, and thus offers an explanation 
of how changes and developments of thought styles can take place (see ibid., 
117; Fleck 1986c, 103). 

Furthermore, although Fleck seems to think that all that is needed to 
account for changes in thought styles are communication processes in science 
and elsewhere (see Fleck 1979, 109f.) and, despite his obvious reluctance to 
admit the possibility that something can be perceived prior to one’s being 
introduced to the relevant thought style that allows a correct interpretation 
of the percept, his mentioning of those anomalies” suggests that percep- 
tion provides us with information that cannot arbitrarily be subsumed under 
concepts suggested by the thought style at hand. The relevant resource to set 
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the stage for anomalies seems to lie in the non-conceptual content of percep- 
tion that becomes a part of the scientists’ observational results. 

Taking the above considerations into account, it seems reasonable to 
assume that both non-conceptual and conceptual contents can be obtained 
via perception, whereas the former seem to come first. They will partly be 
substituted by the conceptual content acquired later. Nonetheless, some infor- 
mation will remain non-conceptual in kind even in our later developmental 
stages. These contents can constitute anomalies within explanatory theories 
and, as such, motivate further testing which might lead to changes in our the- 
oretical approaches. They can therefore bring about progress in science and 
are thus of particular importance in this context. 

Moreover, as Grundmann points out, adopting an externalist position of 
justification, i.e. regarding perception as a reliable process of information 
gathering, also allows us to make knowledge claims based on those non- 
conceptual perceptual contents (see Grundmann 2008, 497). Consequently, 
the thesis can be defended that perception not only fulfils its function as 
an epistemic source if concepts are available for what has been perceived 
so that observation statements can be formulated as reasons in support of 
beliefs, but also if those concepts are missing. In this latter instance, it seems 
reasonable to assume that there is a presumptive right similar to the one 
highlighted by anti-reductionists in the debate about knowledge by testi- 
mony that can be used as a justificatory basis of our perceptual experiences. 
As Scholz points out, such a presumptive right allows the perceiver to 
trust her perceptual abilities as knowledge suppliers as long as there are 
no reasons undermining and defeating this trust, such as indicators of odd 
lighting conditions or of the previous consumption of hallucinatory drugs, 
etc. (see Scholz 2009c, 199f.). 

But it is not only in this sense that the processes of justification of tes- 
timonial and of perceptual beliefs show similarities. As I have argued 
elsewhere (see M6Bner 2010b, ch. 5.5.3), participants in the activity of tes- 
timony become increasingly experienced with individuals as testifiers or tes- 
timonial contexts during their lifetime. Moreover, it is an important part of 
our educational system to acquaint students with the relevant practices in 
their fields of expertise.** A similar developmental process can be assumed 
concerning perceptual experiences. During their professional training, 
people will usually acquire many of the relevant concepts to verify their 
perceptual experiences in scientific observations. This becoming acquainted 
with the relevant concepts as part of the practical training in science is 
elaborately described by Fleck (see Fleck 1986b). In the next section, we 
will come back to the process of learning that Fleck explains. For now, it 
suffices to point out the analogy of development in becoming (a) a trained 
recipient of testimony in a certain domain and (b) a trained observer in a 
particular context. It is this developmental progress of the epistemic sub- 
ject that also allows us to bring in internalist conceptions of justification 
when concepts are acquired. The observer can then be asked to name the 
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reasons that she thinks support her perceptual beliefs precisely — and they 
can also be checked for adequacy by others if necessary. 

Ina nutshell, then, if perceptual experience provides non-conceptual andcon- 
ceptual content in the way argued above, and if it is agreed that the primary way 
to cognitively access the information transmitted via visual representations is by 
perception, it does not seem far-fetched to claim that the same mechanisms are 
also at work in picture perception. Apparently it is these mechanisms that are 
relevant to consider when enquiring how knowledge might be gained via images 
in science and how visual representations can make epistemic contributions to 
scientific arguments. Furthermore, as our line of reasoning moves from the 
more general case, that is, perception in the broader sense, to the special case, 
that is, image perception, these considerations also rule out Crane’s suggestion, 
namely that perception is not a propositional attitude, because apparently, pic- 
ture perception is not propositionally structured either (see Crane 2009). He 
thus argues conversely, moving from the special case to the general concept of 
perception. Yet this seems to be an inappropriate way to defend his thesis. In 
particular, by regarding the issue in converse manner, Crane cannot and does 
not accommodate the artificial sign character of images in his theory. Yet our 
ability to decode visually presented information has undergone certain devel- 
opmental phases because visual representations have become increasingly arti- 
ficial due to newly developed styles of representation, which has already been 
explained by making use of the theory of image games. 

How to consolidate the theses concerning image perception and those 
concerning the apparent artificial character of visual representations will 
be discussed in detail in the next section. The exact implications entailed 
in pointing to the evolutionarily shaped resource of perceptual experience 
when analysing potential epistemic contributions of visual representations in 
science will also be more closely examined. 


4.2.3 Evolutionary merits of perception 


The question under discussion in this section is whether evolutionary processes 
that have shaped the development of human vision can be employed in the 
argument to explain why it might be advantageous to make use of visual 
representations in scientific discourse. The discussion of this topic will bring 
into focus our abilities of pattern detection and our discriminatory cap- 
acities with regard to colours. However, to mention one constraint to the 
attempted reasoning by analogy right at the beginning, it has to be taken into 
account that scientific images have been characterised as signs, i.e. as artefacts. 
Thinking in this way about the nature of visual representations in science 
seems to suggest that image perception and everyday perception do not have 
parity. This apparent difficulty will carefully be considered in the final part of 
this section. 

The capacity of human perception to transmit non-conceptual content 
is explicitly related to evolutionary processes by Peacocke. He states that 
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“[njonconceptual representational content is part of our animal heritage” 
(Peacocke 2001b, 615). Such a line of reasoning arises somewhat naturally 
from his considerations concerning the perceptual abilities of non-human 
animals. Here, Peacocke claims that “[I]f the lower animals do not have states 
with conceptual content, but some of their perceptual states have contents in 
common with human perceptions, it follows that some perceptual representa- 
tional content is nonconceptual” (ibid., 614). Admittedly, even lower animals 
can have perceptual experiences, although they might be different from ours. 
Moreover, we would usually also assume that, in particular, lower animals 
do not possess concepts in the way we do or even not at all. Nonetheless, 
observing the behaviour of those animals suggests that they indeed receive 
relevant information via their perceptual apparatus to react in accordance 
to certain stimuli. Thus, it seems reasonable to assume that those animals 
receive a kind of non-conceptual content via perception (see also Bermudez 
and Cahen 2015, sect. 4.3). 

Yet a qualifying remark has to be added. Our main concern is with visual 
perception, i.e. vision, as the predominant human sense modality. Of course, 
the predominance of vision cannot be so stated regarding other species in gen- 
eral. Some perceptual experiences of animals might be completely different in 
comparison with human sense perceptions. This is due to some animals being 
equipped with sense modalities that human beings do not possess, such as a 
cat’s whiskers or the capability of sharks, hammerheads and sawfish to detect 
the electrical field emitted by their prey. 

Here, some interesting questions arise concerning the connection 
between sense modalities and concepts on the one hand, and concepts and 
communication on the other. How are our concepts influenced by our pre- 
dominant sense modalities? Do our concepts evince more details regarding 
visually perceivable properties (or do we possess more of those concepts?) 
than regarding properties related to audible, tactile and other forms of per- 
ception? Is it because it was more relevant to our ancestors to communi- 
cate, for example, visual clues about prey and predators in their environment 
that our concepts are richer in information regarding visual details than 
other kinds of information? The archaeologist Steven J. Mithen explains, 
for example, how a sensitivity to visual information might have been an 
evolutionary advantage for our ancestors in the context of specialising in 
certain food supplies. 


Knowledge of carnivore behaviour and distribution would therefore 
appear to have been critical to early Homo: competing carnivores may 
have provided both a threat and an indication of a possible scavenging 
opportunity. In this light it would seem improbable that H. habilis [This 
ancestor of the human race existed around two million years ago. N.M.] 
could have exploited the carcass niche if it had not mastered the art of 
using inanimate visual clues, such as animal footprints and tracks. 

(Mithen 1998, 115, my italics) 
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Putting these empirical questions aside, Peacocke definitely has a point when 
relating the thesis that perception can yield non-conceptual content to the evo- 
lutionary development of the human species as visually perceptual beings. In 
the following, the focus will be on our visual abilities and the discussion of some 
consequences for images in science as bearers of information resulting from our 
evolutionarily grounded predisposition to visually access the world. 

The opportunity to gain non-conceptual content via perception, and 
thus to draw on a very rich source of information, may allow us to discover 
anomalies or new phenomena and even to communicate our findings by inte- 
grating images of them in scientific publications. Beyond that, accessing our 
world visually also enables us to make use of certain strategies of informa- 
tion gathering and processing that are particularly shaped by evolution. Thus, 
empirical results from the different fields of research about perception should 
be taken into account. Regarding the analysis of visual representations as 
bearers and providers of information in scientific discourse, I would like to 
suggest two further capacities of the human visual sense modality which 
might alter the balance of the argument when considering the relevance of 
visual representations in comparison with other representational modes. 

Firstly, human observers are obviously good at pattern detection. As 
becomes clear in Mithen’s quotation above, the correct reading of — i.e. 
the correct recognition of behavioural patterns in — animal footprints 
was of such evolutionary advantage to our ancestors that we can assume 
it was one of the causal factors in the development of a certain visual 
expertise in this regard. That, on average, people are indeed particularly 
skilled in pattern detection makes a more or less current trend in science 
quite reasonable, namely the initiatives of the so-called citizen sciences.* It 
seems worthwhile to select and examine just one of these, namely the pro- 
ject called “Galaxy Zoo” (see www.galaxyzoo.org/, accessed January 12, 
2016). Recent sky surveys have yielded huge amounts of data that have 
to be somehow analysed. “Galaxy Zoo” is part of this analytical pro- 
ject. Here, volunteers — particularly laypeople without special training — 
participate in classifying galaxies, photographed by space telescopes such 
as Hubble, according to their shapes. The data are made available online, 
as well as descriptions of the task, including suggested patterns of what the 
objects might look like.*° The user is guided through the classifying task via 
online dialogues. She can accomplish the required classification without 
great effort. All that is needed is an attentive eye to compare the photo- 
graph with the patterns suggested. The project organisers obviously rely on 
their volunteers’ ability to detect patterns. This task cannot be performed at 
the level of numerical data, i.e. as computational operations by IT devices. 
There is a certain vagueness involved in classifying these objects that 
cannot be removed by adhering to numerical data. It can only be reduced 
by consistent results obtained by various human classifiers. That is, the 
same object is categorised by several people to minimise the possibility of 
distorting biases and errors resulting in misclassifications. 


274 The cognitive content 


It seems to be a consequence of evolutionary processes that human obser- 
vers are particularly skilled in tasks like this. Thus, making information avail- 
able in a way that activates these skills, i.e. as visual data that can be investigated 
by vision alone, can enhance our understanding by connecting these abilities 
to the cognitive processing of the respective information. In particular, the 
huge amount of data yielded by today’s IT-based research processes, such 
as measurements recorded as visual data at the LHC in Geneva, the digital 
photographs taken by the Hubble space telescope, or the data collected in 
recent sky surveys undertaken in regard to the mapping of the sky by the 
SLOAN Digital Sky Survey (see www.sdss.org/, accessed January 12, 2016), 
challenge scientists to think of new ways to evaluate these large data sets. 
Maynard thinks that the best new ways to achieve this are sometimes rather 
ordinary old-fashioned ones. “Confronted with the increasingly large data 
sets from its latest cognitive technologies, our species has fallen back upon its 
greatest natural information-processing resources, inherited over millions of 
years: notably those for vision [...]” (Maynard, unpublished paper).*! 

Yet, as the practice of multiple visual classification of the same object by 
different observers shows, scientists are well aware that blind reliance on the 
capabilities of human observers would be problematic, as the virtues of our 
visual skills might sometimes turn out to be vices where science is concerned. 
Hentschel explains this when discussing the capacities of human observers 
(see Hentschel 2014, ch. 9). His main example are the pitfalls that led scientists 
in astronomy seriously astray in the nineteenth century when investigating the 
surface of Mars with their telescopes. Here, he refers to what has become 
known as the ‘Martian canals’. In 1879 the Italian astronomer Giovanni 
Virginio Schiaparelli published a map of the surface of Mars, showing “thin 
lines, apparently interconnecting the various oceans (mares), lakes (lagos) 
and huge rivers (fluvius) [...] In later maps, he incorporated a profusion of 
these thin lines, interconnected in a strange maze-like network mostly origin- 
ating from one of the mares” (ibid., 299). Furthermore, by calling these lines 
‘canali’, which can be translated either (in the more neutral way) as ‘channels’ 
or, in the way most of his contemporaries did as ‘canals’ (which brings the 
artificial character of those structures to the fore), Schiaparelli created an 
illusion which trapped human observers till the end of the twentieth century, 
namely that these observational results show intentionally brought-about 
artificial structures on Mars indicating the presence of some sort of intelli- 
gent inhabitants on the planet. 

However, as Hentschel makes clear, those apparent canals were just 
the result of the poor resolution of the telescopes used at that time and 
a certain tendency in human vision to connect isolated dots into lines 
(see ibid., 307). Moreover, people also seem to have expected to see those 
patterns after Schiaparelli had published his results. Thus, our ability to 
detect patterns relatively easily can also result in unintended interpretive 
impositions of expectations of what is actually visible. Accordingly, 
Hentschel concludes that: 
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[t]he moral of this little tale is that pattern recognition is not an easy 
business. It can easily go astray, especially if the all-too-human obser- 
vers are swayed too much by suggestive analogies to well-known terres- 
trial phenomena. The foregoing is also a lesson about how finicky human 
vision is. It is optimized to resolve potential threats quickly and thus 
evolutionarily programmed to over-interpret rather than under-interpret 
sensory perception. That is why humans are still quicker than modern 
computer programs in detecting visual patterns. But this speed comes at 
a price. Our vision occasionally plays tricks on us. 

(Hentschel 2014, 309) 


Thus, mere reliance on our perceptual abilities alone seems to be not 
epistemically advisable. 

Bearing this in mind, let us turn to the second merit of visual perception 
that evolution has been working on. Belonging to a diurnal species, the human 
eye is particularly sensitive to the spectrum of visible light, that is, to a broad 
range of colours. It has already been noted above that human observers seem 
to be particularly skilled in detecting even subtle differences in colour hue — 
even though we often do not have words to term our observations correctly. 

With respect to this phenomenon, Karl Schawelka offers an explanation 
referring to evolutionary processes in human perceptual capabilities (see 
Schawelka 2007, 64ff.). He draws our attention to the fact that our visual 
apparatus is adapted to natural sunlight. Although it might be objected that 
the spectrum of sunlight is very broad and vast parts of it are simply invis- 
ible to the human eye, Schawelka makes plain that our visual apparatus is 
nonetheless perfectly adapted to its tasks. Because of the filtering effect of the 
atmosphere, not all kinds of solar radiation reach the surface of Earth to the 
same degree. There is a notable maximum within the range of visible light that 
is exactly the part of the spectrum that the human eye is particularly skilled 
in detecting and discerning, i.e. wavelengths between approximately 400 and 
700 nm (see ibid., 65). 

Moreover, Schawelka highlights an interesting biological finding, namely 
that the different visual abilities of our perceptual apparatus cannot be sim- 
ultaneously developed equally well. Apparently spatial perception, colour 
perception and visual acuity cannot be optimal at the same time. Being par- 
ticularly good at one of them also implies a loss in those other visual abilities 
(see ibid., 68). This takes us to the question of what the evolutionary benefits 
were that stimulated the emphasis on colour vision in the evolutionary devel- 
opment of our ancestors. 

A full-blown colour perception appeared as a characteristic of our early 
ancestors around 35 million years ago (see ibid., 74). Schawelka points out 
that being good at discerning different colour hues allowed our ancestors to 
spot, for example, ripe fruits quite easily (see ibid., 72ff.). As omnivores, our 
ancestors certainly enjoyed the evolutionary benefit derived from this cap- 
ability of their visual apparatus. It has to be added, however, that further 
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experiences also seem to be of relevance, as colour alone does not indicate 
digestible food per se. Ripe apples and fly agarics, for example, share the same 
colour, namely red — a particularly salient colour to the human eye (see ibid., 
72)—but this does not imply that we are advised to taste the latter. On the con- 
trary, colours can also indicate poison — not only regarding mushrooms and 
plants, but also insects, fish, snakes and amphibians, such as South American 
tree frogs. Apparently we also profit from our visual abilities to discern 
colours easily in this case if the relevant background knowledge is available to 
us. The colourful skin of those animals will make us deal with them carefully 
or simply avoid them as soon as we spot them.” 

John Campbell connects these more general ideas about the evolutionary 
merits of colour perception to human capabilities of reasoning when pointing 
out that “[i]t is often observed that colours of objects have predictive value. 
The particular colours of various foods are predictive of their nutritional 
value. The exact colours of particular people and plants are good predictors 
of their health” (Campbell 2009, 35). Campbell here not only highlights the 
role of colours as indicators of certain properties, such as ripeness or toxicity, 
but also draws our attention to the way in which our capacity to visually dis- 
cern colours can be useful in science. In medicine, the colour of human skin 
can, for instance, be an indicator of certain diseases such as yellow fever or 
tuberculosis. 

Sabine Miller and Dominik Groß discuss the usage of false colour 
images in medical science and in astronomy (see Miller and Groß 2006). 
They point out that this kind of visualisation is used especially for two 
purposes in those domains: firstly, to make visible what is otherwise not 
accessible to the human eye, but can be detected by other means such as 
ultraviolet radiation (see Figure 2.2 as an illustration of this point) and, 
secondly, for supporting the interpretation of images by enhancing their 
contrast (see ibid., 94). This latter purpose draws explicitly on the ability 
of human observers to discern between different colours — for example, 
the photograph of Olympus Mons (see Figure 2.12), in which false colours 
allow us to identify the different altitudes of the volcano in its different 
regions without great effort. In comparison with this, extracting the same 
information from a plain photograph or a greyscale image would be much 
more difficult. Consequently, as colours are used to transmit information 
and apparently support recipients in grasping it relatively easily, Müller and 
Groß defend the thesis that colours can be regarded as epistemic tools (see 
ibid., 114; Groß and Müller 2006, 79). 

Despite these advantages of colour vision, it has to be added that, without 
going into all the details of the debate, there is a particular difficulty related to 
suggesting colours as scientifically valid indicators of certain states of affairs. 
Although human observers are usually very good at distinguishing between 
different colours, the latter cannot be regarded as objective properties of the 
entity to which they seem to be attached. This problem is also addressed in 
Campbell’s text: 
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[m]any philosophers — the classical sources are Galileo and Locke — have 
said that science shows that there is a mistake embodied in our ordinary 
understanding of colour concepts. We commonsensically take colours to 
be categorical properties of objects, whose nature is apparent to us in 
vision, but in fact there are only complex microphysical structures and the 
consequent tendencies of objects to produce ideas in us. 

(Campbell 2009, 40) 


What is highlighted by those scholars mentioned by Campbell is that colour is 
not an intrinsic property of objects, but a phenomenon that is solely present 
in our perception of those objects. Yet in science we want to find out what 
the entity under investigation is like, and are not so much interested in the 
dispositions of human vision to interpret its percepts in certain ways. 

What makes matters of colour perception so complex requires two brief 
explanatory remarks. Firstly, scientists can detect wavelengths of radiation 
and so measure which parts of the spectrum have been absorbed and which 
have been reflected by an illuminated entity. Wavelengths are commonly 
associated with certain colours. Spectrographic analyses in astrophysics, for 
example, make use of this nexus by assessing the absorption lines of certain 
kinds of gas in a given spectrum emitted by a particular celestial object to 
determine the ingredients of a particular nebula or star, yet physicists are 
reluctant to regard colours inferred from wavelengths of (reflected) radiation 
as characteristics inherent in those objects. Glenn Elert, for example, states 
bluntly that: 


[c]olor is a function of the human visual system, and is not an intrinsic 
property. Objects don’t ‘have’ color, they give off light that ‘appears’ to be 
a color. Spectral power distributions exist in the physical world, but color 
exists only in the mind of the beholder. 

(Elert 2014, ch. 3.3.7) 


The second aspect that makes matters even more complex is that our 
perception of colours is highly influenced by a variety of factors outside 
the assumed bivalent relation between our visual system and the object 
perceived. What kind of light illuminates the object in question — a warm 
candlelight or a cold light emitted by a fluorescent tube? What are the 
surrounding colours like that the object is embedded in? Dark context- 
colours let us perceive a brighter hue of the object’s colour than light 
context-colours do. Schawelka explains these puzzling phenomena of 
colour perception in detail. He highlights the fact that context-colours 
can either contribute to our experience of colour consistency or, to the 
contrary, trigger the phenomenon known as ‘simultaneous contrast’, that 
is, two physically identical stimuli that can result in two totally different 
colour experiences depending on the context-colours of these stimuli (see 
Schawelka 2007, 49). It therefore seems rather difficult to refer to our 
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ordinary perceptual abilities to detect certain colours as a reliable tool of 
scientific analysis. In order to achieve reliable results, what seems to be 
required is the use of instruments unaffected by such distorting visual phe- 
nomena; instruments that can deal with these problems by abstracting the 
objective value from those additional factors.“ 

To sum up my results so far, the above considerations suggest that evolu- 
tionary processes have provided human beings with particular visual skills. 
On average, we are able to gather a wide range of information via visual per- 
ception, even if this is non-conceptual in kind. Moreover, nature has endowed 
us with certain detective abilities, especially with respect to grasping patterns 
and colours, that have enabled us to quickly focus our attention on aspects 
of vital interest in this wealth of information. Those abilities were apparently 
advantageous to the survival of our ancestors — and can still be useful in the 
everyday context as well as in the sciences. 

The usefulness of those skills in epistemic contexts is defended by edu- 
cational psychologists. In particular, Eleanor Gibson’s work on perceptual 
learning is based on the discriminatory abilities of human sense perception. 
Benedict Carey explains how learning is considered in this context. He writes: 


[p]lerceptual learning is active. Our eyes (or ears, or other senses) are 
searching for the right clues. Automatically, no external reinforcement 
or help required. We have to pay attention, of course, but we don’t need 
to turn it on or tune it in. It’s self-correcting — it tunes itself. The system 
works to find the most critical perceptual signatures and filter out the 
rest. 

(Carey 2014, 184) 


Carey points out that quite a few computer-based learning programs take 
advantage of perceptual learning. Such tools are, for example, used in flight 
training to teach trainees “perceptual intuition” (ibid., 188). The dials of 
six main instruments in the cockpit of a small aircraft are displayed on a 
computer screen, and the trainee is asked to choose between seven different 
options of what these instruments tell her in sum about the position and 
activity of her plane. She is thus trained to make her choices just by looking 
at her instruments and not by considering each of them in detail, trained to 
develop a gut feeling, so to speak, about what those instruments in sum tell 
her, as flying means multitasking, especially during landing manoeuvres. That 
is, the pilot not only has to read her instruments, but also talk to the tower, 
consider visual information, perform the necessary tasks — and all of this 
more or less simultaneously. Thus, the pilot simply does not have the time to 
read and interpret each instrument separately, but needs to base her decisions 
on a quick look at them. 

This example of training tools drawing on perceptual learning makes clear 
that human observers are particularly good not only at discriminating between 
relevant perceptual clues, but can also become even better at this task through 
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regular visual training. Carey’s examples thereby support the suggestion 
that, because picture perception draws on these evolutionary resources, using 
visual representations as a means to transmit information in scientific dis- 
course allows us to profit from these virtues of our visual apparatus. 

Moreover, this line of reasoning offers a rationale for the epistemic qual- 
ities of images of various kinds, as pointed out in Perini’s work, for example. 
She shows that, for instance, micrographs can inform us about spatial features 
of the specimen, even if we do not have concepts to verbally describe what we 
have observed by inspecting the image. She claims that these images 


can represent unfamiliar phenomena, without the need to articulate 
hypotheses about results prior to the experiment. As in this case, fig- 
ures that support the existence of unexpected structural features can 
be produced. This system can also represent very complicated struc- 
tural properties, even when there are no linguistic terms for the same 
features. 

(Perini 2005c, 921) 


In a similar fashion, Kulvicki’s claim that images can transmit vast amounts 
of information across various levels of abstraction more or less immediately 
(see Kulvicki 2010b) can be explained with recourse to the epistemic merits of 
the perceptual abilities of human observers. 

Caution is definitely advisable, however, that optimism about similar- 
ities between picture perception and perceptual processes in general is not 
misleading us when analysing the epistemic capabilities of scientific images. 
Maynard stresses that there are important differences between both percep- 
tual processes (see Maynard 2011). He highlights the fact that images* are 
artefacts. Acknowledging this implies that it has to be taken into account that 
people are accustomed to handle artefacts in certain ways. This means that 
they regard visual representations as being produced for certain purposes, 
which they then try to identify when looking at images. This is why Maynard 
thinks that picture perception is, at least to a certain degree, different from 
visual perception in everyday life. Moreover, this crucial distinction is also his 
rationale for rejecting an unqualified application of theories of visual percep- 
tion to picture perception. He writes that: 


‘[vjisual array’ treatments of pictures dominant in philosophy and 
perceptual psychology ignore these basic features about perception. 
Overlooking that depictions are familiar artifacts and used as such, they 
neglect the great differences between looking at things and looking at 
depictions of them. Notably, they hide the fact that we look at depictions, 
not real scenes, for their depictive ‘for’ affordances, and in terms of ‘on 
purpose’: why they were put or left there, what they are doing there — 
including whether they should be there. 

(Maynard 2011, 21) 
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Thus, although the deciphering of visual representations in science might 
benefit greatly from being based on perceptual processes that allow the investi- 
gator to rely on cognitive mechanisms especially shaped by evolution, Maynard 
is obviously right in reminding us of this difference between ordinary and pic- 
ture perception. Following his advice seems to be most crucial with regard 
to the initial thesis, namely that visual representations in science can best be 
understood as signs. However, acknowledging this artificial character of images 
in science does not undermine the possibility of drawing on the cognitive 
resources provided by visual perception. An explanation is due at this point. In 
section 4.1.3, and with reference to Scholz’s concept of image games, we already 
discussed the possibility of a layering of meanings in visual representations; 
that is, the addition of new layers of meaning to its initial encoding by making 
use of images in communicative contexts. It is here that interpreters of sci- 
entific images will be most attentive to their object’s artificial character and 
question the intentions of those people who either produce or distribute those 
images. It was suggested that strategies of justification developed in the debate 
about knowledge by testimony concerning the epistemological role of visual 
representations in scientific discourse be employed to deal with this situation. 
Those models of justification are primarily based on assumptions about the 
speaker and only secondarily on considerations about the content transmitted, 
such as its coherence with background beliefs. Thus, Maynard’s reminder does 
not pose a counter-example to the above line of reasoning. 

As a concluding remark, I want to highlight another finding of Maynard’s 
analysis concerning the cognitive content of visual representations. He 
emphasises that adhering to the results of perceptual sciences too strictly 
might lead to an unwanted consequence concerning the epistemic status of 
(scientific) images, namely that they are regarded as external to the human 
mind, i.e. as objects of visual investigation similar to natural entities (see 
Maynard 2011, 11). Such an externalist conception apparently speaks against 
the possibility of using images as signs that can be purposefully produced to 
transmit information. Contrary to this, no problems of this kind have ever 
been asserted with respect to language. Maynard not only notes this discrep- 
ancy between estimations of the epistemic capabilities of images and of verbal 
language on the part of scholars relying on perceptual sciences to explain 
the epistemic accomplishments of visual representations, but also traces this 
stance in the analyses of evolutionary palaeoanthropologists who theorise 
about the origin of the human mind and the capacities of our ancestors to 
communicate via symbolic means (see Maynard, unpublished paper). 

Maynard points out that although scholars of this profession often use 
cave paintings as evidence of our ancestors’ developing capabilities to com- 
municate by using symbols, they more or less immediately deny visual 
representations their epistemic roles. 


Returning to the standard image-to-language reasoning, it is interesting 
that once we have used image thought-content to infer the advanced 
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linguistic kind in the Ice Age, we put aside and even deny the former. 
In Philosophy, as in science, the Paleolithic evidence is swiftly awarded 
the honorific title ‘art’, allowing us to treat it as something outside the 
mind [...] 

(Maynard, unpublished paper) 


Such a negative stance towards images as content providers is expressed 
for example by David Lewis-Williams, an expert on cave paintings. He 
writes that: 


[t]he best that can be said for pictures is that they trigger memories of 
information that has been absorbed in different ways, that is, by experi- 
ence and verbally. So whilst the Upper Palaeolithic images may have 
sometimes functioned as mnemonics, their capacity to store or convey 
information was limited. They were not like the hard drives of modern 
computers. 

(Lewis-Williams 2004, 67) 


Without doubt, Lewis-Williams suggests an epistemically inferior status for 
those visual representations produced by our ancestors in comparison with 
their first verbal utterances. He particularly denies that those images can 
possess a kind of cognitive content similar to linguistic expressions that allows 
the storage and transmission of information. 

However, there are other scholars working in this field who see less of 
a discrepancy between the epistemic capabilities of linguistic and visual 
representations. Mithen’s opinion about this topic offers, for example, an 
interesting contrast to the sceptical attitude diagnosed by Maynard. About 
the contributions of those early visual representations to the information pro- 
cess he states that: 


[i]n summary, although the specific roles that prehistoric artifacts may 
have played in the management of information about the natural world 
remain unclear, there can be little doubt that many of them served to 
store, transmit and retrieve information. Major benefits of this will have 
been enhanced abilities to track long-term change, to monitor seasonal 
fluctuations and to devise hunting plans. Many of the paintings, carvings 
and engravings of Modern Humans were tools with which to think about 
the natural world. 

(Mithen 1998, 197, my italics) 


I agree with Maynard that the link between those early visual 
representations and language as tools of communication is apparently better 
understood along the lines of Mithen’s reasoning than of Lewis-Williams’s. 
What reasons are there to deny that those images were meant to transmit 
information encoded by those early artists? 
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Taking this as background information for the analysis, it seems reasonable 
to assume that visual representations can bear a cognitive content and thus 
play an epistemic role in communication. Moreover, acknowledging this fact 
also prepares for taking into account the long history of this practice of visual 
communication. That is, long before mankind developed symbols to record 
language, a way to transmit information visually via those images had been 
established. Moreover, not only does visual communication greatly pre-date 
the use of writing skills, it also has lasted much longer. Our ancestors made 
use of this practice for around 30,000 years, as Lewis-Williams claims: “[s]till, 
radiocarbon dating strongly suggests that the Upper Palaeolithic period, that 
part of the Palaeolithic in which people began to make art, lasted from about 
45,000 to 10,000 years ago” (Lewis-Williams 2004, 39), and we still use images 
in our communicative habits today. Thus, comparing this rather impressive 
time-span with the few thousand years of our practice of writing, it should 
come as no surprise that human beings had already developed considerable 
skill in encoding and decoding visual information. Thus, it is not only the evo- 
lutionary development of human vision that allows us to benefit epistemically 
in processes of deciphering visually presented information, but also this long- 
term social practice. 


4.2.4 Interim results: what can be learnt from theories of perception? 


The above discussion of the cognitive content of visual representations 
in science was motivated by Perini’s initial observation that it seems to be 
common practice in scientific discourse to use images when attempting to 
transmit essential information about research results, hypotheses and meth- 
odology. Hentschel points out that there are 14.8 images per ten pages of text 
in contemporary scientific journals in biology and 12 images per ten pages of 
text in physics journals (see Hentschel 2014, 30). It seems, then, that scientists 
share a particular preference for making use of visual representations to 
transmit certain information. 

Despite this widespread practice, philosophers question the epistemic cap- 
abilities of visual representations in scientific discourse, as Perini has made 
clear. In particular, they express reservations about the suitability of visual 
means to be regarded as proper components of scientific arguments — whereby 
‘proper components’ refers to the assumed epistemic capacities of images. The 
above analysis was thus intended to inquire more deeply into the prerequisites 
of these capacities and, especially, the question about the precise nature of an 
assumed cognitive content of scientific images had to be answered. 

With regard to the epistemic capabilities of visual representations in the 
context of scientific arguments, proponents of the traditional Fregean point 
of view seem to regard ‘argumentation’ as an exclusively verbal activity. Their 
main objection, then, consists in the fact that images are apparently not 
propositionally structured. As their line of reasoning suggests that epistemic 
functions are tightly bound to propositional contents alone, they think one is 
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seemingly entitled to disregard visual representations as epistemic means in 
this context. 

Contrary to this, a different account has been presented here, as it can be 
assumed that Perini is correct in pointing out that scientists handle those visual 
elements differently from the way proponents of the Fregean point of view would 
suggest when affording them a role in scientific discourse, and that it would be 
wrong to disqualify such practices from an epistemological point of view. 

The first suggestion, already argued for in the last section, stipulated 
starting with a different account of ‘argumentation’, namely to regard argu- 
mentation as a communicative act that can be identified via a certain intention 
of the speaker and which can be performed by using different representational 
means. Stripping off the too narrow conception of mere verbal utterances 
has made it possible to take into account the epistemic achievements of 
visual representations in the context of scientific discourse more seriously. 
Apart from Kjorup, whose work we have used as a guideline in this respect, 
other scholars — in particular those working on argumentation theory — have 
suggested similar, albeit more broadly conceived methodological approaches 
to the topic of argumentation.” 

Yet, although this initial shift in perspectives allows us to take epistemolog- 
ically seriously the phenomenon of visual representations, the first question 
that still has to be answered refers to the kind of cognitive content such images 
might contain and are thus able to transmit in the argumentative context. In 
this respect, the point has already been stressed that we are apparently able to 
translate contents from one representational means into another. This holds 
true also in the case of images and therefore speaks in favour of the thesis that 
at least some of them contain a cognitive content that can be transmitted in 
communicative acts. 

The question of translatability has triggered another critical point, namely 
whether scientific images are only auxiliary means in the distribution of con- 
tent, i.e. whether the choice of a visual presentation of information is only due 
to the weakness of the human mind to grasp the data otherwise.“ Philosophers, 
highlighting this (in principle) reducibility of visually transmitted informa- 
tion, suggest that visual representations do not contribute to the acquiring 
of scientific knowledge or other epistemic desiderata, although, from a prac- 
tical point of view, images may be preferable to other kinds of representations 
in science. Irving, for example, mentions this sceptical philosophical stance 
appearing in the discussion about the epistemic value of images in scientific 
arguments (see Irving 2011, 775). 

The suggestion to reduce visually presented content to a presumably 
epistemically more basic category is critically commented on by Perini. She 
refers to the reducibility thesis as a putative rationale for why philosophers of 
science apparently take it for granted that visual representations are not worth 
analysing as epistemic tools. As Perini states, this neglect reveals “an under- 
lying assumption: The reasoning involved when scientists support hypotheses 
with figures can be understood without considering visual representations as 
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such” (Perini 2005c, 914). However, neither Perini nor I are convinced by this 
line of reasoning, as it completely ignores scientists’ preferences for visual 
representations in their reasoning practices. 

The question of how to argue for an independent epistemic status of scien- 
tific images then took us back to the debate about propositionality. If we want 
to defend the thesis that visual representations can yield knowledge or other 
epistemic desiderata, we apparently have to discuss first what kind of con- 
tent — propositional or non-propositional — they can contain and transmit. In 
the response to this query, we focused on the fact that it is via perceptual means 
that we decipher the content of scientific images. Apart from the sceptics, 
most contemporary philosophers agree that perception can be regarded as an 
epistemic source, i.e. as a source of propositional knowledge, thus, pointing 
to the fact that cognitively accessing the content of images via vision also 
permits drawing an analogy to everyday perception. That is, if it is admitted 
that perception in those everyday contexts yields propositional knowledge, 
then this epistemic mechanism should not be called into question when sci- 
entific images are visually investigated. The view has therefore been endorsed 
that at least some visual representations can transmit a propositional content. 

Yet a more detailed look at the topic has made clear that there is more to 
the epistemic dimension of visual representations than simply propositional 
knowledge via perception. In particular, Hentschel’s example of the first 
observations of the solar surface has made plain that our visual capabilities 
are not restricted to already-conceptualised domains. On the contrary, human 
observers are apparently able to detect vast amounts of information via their 
visual apparatus, even though they do not possess relevant concepts to name 
and describe correctly what they have observed. This is not only true with 
regard to people presumably lacking the relevant individual training in tech- 
nical vocabulary, it also holds true even if there are no suitable concepts avail- 
able per se — as in the case of the images produced by electron micrographs 
discussed by Perini. 

In this sense, I also partially agree with Grundmann’s thesis that our percep- 
tual apparatus allows us to access non-conceptual and thus non-propositional 
content as well. This seems to be how perception genuinely works. Concepts 
to classify what we see are often acquired when other people — parents or 
teachers — explain to us what we have observed. Thus, those concepts are 
acquired via testimony, that is, by the words of others, that accompany, as 
explanations, observational situations particularly in our early years. Here, 
testimony and perception work hand in hand to form the basic knowledge on 
which our belief system is built. When the relevant concepts are acquired in 
the way described, it does not seem so far-fetched to assume that we are also 
able to gain propositional knowledge via perception.** The suggestion, then, is 
that some visual representations can transmit both kinds of contents — prop- 
ositional and non-propositional alike. In particular, representational images 
such as photographs or instrumentally produced images such as micrographs, 
brain scans, or computer graphics, seem to let the observer benefit from her 
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perceptual ability to detect and decipher non-conceptual as well as conceptual 
contents. 

Finally, taking into account that it is via perception that information, trans- 
mitted by visual representations, is cognitively accessed, has also made plain 
why we are obviously particularly skilled in decoding information presented 
in this way. Empirical evidence supports the thesis that visual perception has 
developed as a cognitive resource for human beings that has been especially 
shaped by evolutionary processes. In accordance with this, Hentschel claims 
that “[a]pproximately 60% of the input into the human brain comes from 
vision” (Hentschel 2014, 32). Not only do we gain huge amounts of informa- 
tion via our perceptual apparatus, even if we do not possess relevant concepts 
of what we see, but human vision has also evolved in such a way that we are, 
for example, particularly good at pattern recognition or detecting colour hues. 

Therefore, choosing visual representations to transmit information in 
scientific discourse also means attempting to benefit from these cognitive 
advantages provided by those evolutionary processes. Diagrams and graphs, 
for example, are particularly useful to exploit our ability of pattern recogni- 
tion. Photographs and other images brought about by causally functioning 
instruments are especially suitable as evidence, because they can transmit 
information otherwise only available to the initial observer — the eye-witness, 
so to speak. Furthermore, acknowledging the capacity of images to transmit 
even non-conceptual content enables scientists to draw on the further merits 
of visual representations. On the one hand, scholars can start communicating 
about phenomena that have not yet been completely conceptualised. That 
is, visual representations allow scientific collaborative investigations even 
when no clear concepts of the phenomena are available. They not only allow 
working out the details, but also the elaboration of the initial concept itself. 
On the other hand, the fact that images can transmit non-conceptual contents 
also explains how it might be possible that such visual representations can 
be used to refute certain hypotheses in science. Their content might reveal 
to subsequent observers anomalies or new phenomena not expected by the 
initial investigators. Photographs taken by astronomers can illustrate this 
point. Ratzka explains that, although those images were initially not intended 
for this purpose, early photographic recordings are re-evaluated nowadays 
to check for detections of the movements of asteroids — a phenomenon not 
known to those early astrophotographers (see Ratzka 2012, 246). Used this 
way, those pictures refute the initial hypothesis that all the detected entities 
were stars. 

All of these aspects add to the explanation of why many scientists favour 
visual representations in their communication processes. Moreover, it also 
suggests an explanation for the reason that certain academic disciplines are 
more attached to such a visual method of information transmission than 
others: in particular, empirical sciences apparently profit from the evidential 
role that visual representations can play in scientific discourse. Moreover, sci- 
entific disciplines that produce huge quantities of numerical data — as the 


286 The cognitive value 


results of measurement or simulation processes — benefit from the human cog- 
nitive capacity to evaluate visualisations rendered from those data sets. 

The final remark in this summary is meant to bring into focus a crucially 
divergent aspect between everyday and picture perception as highlighted by 
Maynard. He urges us to keep in mind that it would be wrong to claim a com- 
plete identity in this respect. Most of all, we have to be aware of the fact that it 
is a part of the recipients’ background knowledge that visual representations 
are artefacts, i.e. that they are created and circulated within the community 
for certain purposes. Scientists will therefore usually take into account the 
intentions of the communicating party when evaluating visual representations 
in scientific discourse. 

The suggestion, in this regard, was to make use of Scholz’s account of 
image games. This approach explains how a layering of meanings can take 
place, that is, how initially encoded information can be enriched by the 
intentions of different people making use of the genuine image in diverse 
communicative settings. In this way, the apparent tension between regarding 
scientific images as signs and as entities subjected to a basically perceptual 
decoding can be relieved. Moreover, as the theory of image games allows us 
to endorse the artificial character of scientific images despite their percep- 
tual decoding, we can make use of their sign character to help explain why 
certain visual representations are more easily understandable than others. 
Some — such as photographs — are more closely related to our capabilities of 
perceptual decoding than others, for example diagrams whose interpretation 
requires more training, that is, background knowledge. 

Having laid out my hypotheses on the relations between the perceptual 
abilities of human observers, the cognitive content of visual representations 
and the functional roles of images in scientific discourse, I will next discuss 
what effects these findings might have on an epistemological evaluation of 
scientific images. 


4.3 The cognitive value of visualisations 


The above considerations about the perceptual deciphering of visually 
presented information suggest two points about the epistemic status of scien- 
tific images. Firstly, the thesis has been defended that at least in those instances 
where the relevant concepts are at the observer’s disposal, propositional know- 
ledge can be gained via picture perception. Moreover, the acquisition of these 
concepts is often made possible partly by our perceptual abilities. Secondly, 
the discussion of non-conceptual contents being grasped via perception and 
the correlated capacity of some visual representations to transmit such a kind 
of information suggests that, from an epistemological point of view, there is 
more to the epistemic status of these visual means than being sources of prop- 
ositional knowledge alone. 

In sum, the above analysis shows that claiming a necessary inferiority of 
visual representations in comparison to other representational means when 
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information transmission is at issue — a claim that proponents of a traditional 
Fregean point of view might defend — is not justified. On the contrary, in the 
previous discussion it was pointed out in what sense images might entail a 
cognitive content which can be used for such knowledge-orientated communi- 
cative purposes in science. Acknowledging the fact that ‘visual representation’ 
is a broad category of diverse visual phenomena which might also be of mixed 
qualities when it comes to transmitting their content to a respective audience, 
it seems reasonable to assume that by and large visual representations have 
parity with other communicative vehicles expressing thoughts. 

In this final section, the focus will be on the question of whether visual 
representations are particularly suitable to serve specific epistemic functions 
in science. Can the thesis be defended that, at least in certain instances, images 
can be regarded as epistemically superior to other representational means 
when it comes to fulfilling particular communicative tasks?” 

In the following, the question of the precise cognitive value of visual 
representations in scientific discourse will be discussed. Is there a kind of 
epistemic advantage inherent to visual representation not present in other 
vehicles of communication? To find an answer to this, the role of images in 
learning activities in science, i.e. within a paradigmatic epistemic process, will 
be examined, whereby the concept of learning is meant here in a broad sense 
and not merely confined to students’ education. On the contrary, learning is 
commonly understood as an essentially cognitive activity.©’ Being successful 
at learning something normally implies two important epistemic desid- 
erata: knowledge and understanding (see Kosso 2007, 175). 

Both of these epistemic desiderata will be discussed separately in the 
following analysis. The investigation is guided by the questions of how visual 
representations fit into the epistemic process of learning, what exactly their 
contributions in this context are, and how they can support the cognitive aims 
of learning. In this context, some of the insights will be applied that have 
been gained concerning the cognitive content of visual representations and 
the perceptual way they are utilised so that their content is understood. That 
some visual representations can transmit non-propositional content provides 
a resource to take into account suggestions discussed in epistemology beyond 
the traditional point of view, which focuses solely on propositional knowledge. 

Firstly, with regard to knowledge acquisition, this means that the 
question to be asked is in what sense visual representations can contribute to 
(1) knowledge-how or to the attainment of (2) mental images via phenomenal 
knowledge or via knowledge by acquaintance. Both ways allow for know- 
ledge representations beyond propositionally structured contents. Moreover, 
philosophers traditionally assume that a disregard for these kinds of know- 
ledge in epistemological analyses is justified because, among other things, 
they can neither be transmitted in conversation nor can logical operations be 
utilised to process them further (see Grundmann 2008, 86). If it can be shown, 
however, that they are indeed communicable, namely by means of visual 
representations, this would not only add to the cognitive value of images in 
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scientific discourse, it might also return into focus those other kinds of know- 
ledge in epistemology. 

The second dimension of successful learning, namely understanding, is related 
to a discussion in epistemology similarly motivating a critical dispute about the 
exclusive focus on propositional knowledge. Triggered by the struggles of ana- 
lytical philosophers to come up with a convincing concept of knowledge in 
the aftermath of the Gettier-cases, which undermined the traditional analysis 
of ‘knowledge’ as justified, true belief, philosophers started discussing what is 
really to be appreciated as valuable with regard to knowledge by advancing 
the questions of the epistemic value of knowledge in comparison to mere true 
belief and whether there are other epistemic desiderata epistemically worth- 
while to consider. In this context, the suggestion has been put forward to regard 
understanding as an epistemic desideratum with an intrinsic epistemic value 
(see Kvanvig 2003, 186). Some philosophers even propose to replace ‘know- 
ledge’ with ‘understanding’ in epistemological analyses. Thus, discussing how 
visual representations can facilitate scientific understanding, would also mean 
showing how they can contribute to an epistemic desideratum valuable in its 
own right, that is, independent of propositional knowledge. 

Considering possible contributions of visual representations to both the 
acquisition of other kinds of knowledge and to the facilitation of understanding 
in science therefore means investigating the cognitive value of images beyond 
the classical setting of the distribution of propositional knowledge. 

To attain a better understanding of the topic, the analysis will begin with a 
discussion of approaches from the realm of educational psychology, in which 
scholars investigate the effects of visual representations on learning processes 
from an empirical point of view. Choosing this empirical perspective as a 
starting point for subsequent epistemological discussions has the advantage 
not only of showing exactly how images can play a part in the cognitive pro- 
cess of learning, but also of highlighting the constraints on their efficiency in 
this context, constituted for example by learners’ characteristics. 


4.3.1 Educational psychology 


In this section, some theses will be examined concerning the effect of 
visual representations on learning processes put forward by educational 
psychologists. A comprehensive overview of this topic is presented by Ioanna 
Vekiri (see Vekiri 2002). She discusses three different theoretical approaches 
from the realm of educational psychology to explain the contributions of 
graphical displays to students’ learning processes. Regarding the educational 
merits of these visualisations, there are, on the one hand, those theories 
dealing with the positive effects on remembering information (see ibid., 262). 
Accounts on dual coding and conjoint retention belong to this set of theories. 
On the other hand, there are approaches, subsumed under the heading of the 
visual argument hypothesis, dealing with the transmission and processing of 
information offered visually. 
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Vekiri focuses her analysis exclusively on graphical displays such as diagrams. 
It can be assumed, however, that other kinds of visual representations show 
similar effects on learners’ cognitive processes. I will point out possibilities of 
an application to a wider range of images in the discussion of the different 
theoretical approaches below. Furthermore, we will focus on dual coding and 
visual arguments accounts alone, as the theory of conjoint retention does not 
add a new dimension to the topic of enhancing students’ cognitive processes. 
It is based on the dual coding approach and applied to the realm of maps, 
thus constituting a case rather of application than a completely new theoret- 
ical approach, as Vekiri makes plain (see ibid., 292). 

Proponents of the theory of dual coding suggest that there are two different 
cognitive subsystems in the human mind: one to process and store verbal 
information and another to process and store non-verbal, in particular visual, 
information (see ibid., 266). Vekiri points out that, despite this assumed 
duality, proponents of this theory nonetheless argue for a linkage between 
both systems. “Although the two cognitive systems are functionally distinct, 
they are interconnected. Associative connections can form between the verbal 
and visual representations, enabling the transformation of each type of infor- 
mation into the other” (ibid., 267). This connection between both systems is, 
for example, used to explain why people are able to mentally visualise certain 
events read in a novel. Moreover, Wolfgang Schnotz defends the thesis that 
the theory of dual coding not only affects graphical displays but also pictorial 
representations such as photographs (see Schnotz 2002, 107). 

A consequence of the theory of dual coding for educational purposes 
consists in the thesis that it is advantageous to present information both visu- 
ally and verbally in this context, for example by adding visual illustrations to 
a text. Two explanations are offered why such a combination of representa- 
tional means can enhance the student’s cognitive process of learning if certain 
design criteria are met. 

The first aspect concerns the retrievability of information processed and 
stored in such a dual way. Vekiri describes the potential positive effect on 
learning processes as follows: 


[ijllustrations and other visual materials may contribute to the effective- 
ness of instructions by enabling students to store the same material in two 
forms of memory representations, linguistic and visual. When verbal and 
visual information is presented contiguously in time and space it enables 
learners to form associations between visual and verbal material during 
encoding.*! This may increase the number of paths that learners can take 
to retrieve information because verbal stimuli may activate both verbal 
and visual representations. 

(Vekiri 2002, 267) 


Obviously, this dual method of cognitively storing and accessing informa- 
tion can be valuable not only in the educational setting that Vekiri discusses, 
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but also in the cases of measurement data shown in diagrams and discussed 
in the related text in a scientific article or textbook, for instance, where the 
very same effect can obtain, namely that two different memory traces might 
support the process of learning the relevant results. 

A second way in which the dual coding theory can be used to explain an 
enhancement of students’ learning processes, when visual and verbal infor- 
mation is presented in the right manner, concerns a reduction of the cog- 
nitive load on the working memory, presumably achieved by offering visual 
information. 


Dual coding theory claims that visual representations can be accessed 
as a whole and processed in a simultaneous manner, whereas linguistic 
representations are hierarchically organized and processed sequentially, 
one piece of information at a time. It is likely that graphics can improve 
our memory of verbal material because, owing to working-memory 
limitations, their mental reconstruction allows faster and more effective 
processing than does verbal representations. 

(Vekiri 2002, 279) 


Again, such a positive effect can be assumed to obtain in the scientific setting 
as well as in educational contexts. 

Apparently the theory of dual coding is a very successful approach to 
explain learning processes. Vekiri points out that recent studies in neuro- 
psychology and cognitive science seem to have proven the theses put for- 
ward by proponents of the dual coding theory empirically (see ibid., 267ff.). 
However, this theoretical approach can account for the relevance of visual 
representations in the epistemic processes of science to a certain degree only. 
The following two reasons make especially plain the minor explanatory status 
of dual coding approaches in this realm. 

Firstly, the dual coding theory somehow presupposes a certain redun- 
dancy in information presentation, i.e. the described positive effect is 
achieved best if visual and verbal information overlap significantly. This 
way of presenting information, however, cannot always be expected to 
obtain in scientific discourse. Adhering to dual coding by all means would 
undermine Perini’s thesis that scientists use visual representations as proper 
components of premises and conclusions in scientific arguments. The crux 
is that Perini’s argument presupposes (and I have also tried to show this in 
the above discussion) that certain kinds of information — for example non- 
conceptual contents — cannot be transmitted by verbal representations. 
Thus, not all kinds of information can be as equally well-presented by visual 
and linguistic representations as is apparently presupposed by proponents 
of the dual coding theory. 

Secondly, the theory of dual coding has an emphasis on remembering and 
ways of retrieving information from our cognitive system. These are, admit- 
tedly, important aspects of learning. However, there are further aspects of 
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visual representations in scientific communication and scientific discourse that 
draw on cognitive abilities not covered by dual coding. As an example, the pos- 
sibility of detecting anomalies undermining prevalent scientific hypotheses to 
explain certain phenomena was pointed out above. Moreover, scientific images 
often serve as evidence to support certain theses in science and are, as such, 
subjected to critical investigations by an audience. None of these functions is 
particularly related to memorising or retrieving information, that is, the sub- 
ject matter of the dual coding theory. Consequently, the theory of dual coding 
is only of partial interest when the epistemic status of visual representations in 
epistemic processes in science is scrutinised. 

The more interesting account in the present context is the so-called visual 
argument hypothesis. Here, the label ‘visual argument’ might be slightly 
misleading as proponents of this approach are not concerned with arguments 
in the philosophical sense. Thus, we are not discussing the validity or structure 
of arguments, namely premises, conclusions and inferential reasoning, rather 
this psychological approach focuses on the ability of visual representations to 
transmit information and to enable the recipient to grasp complex relations 
existing among them. 


Visual argument concentrates on the perceptual and interpretation 
processes that take place when learners extract meaning from graphical 
representations. It claims that graphical displays are more effective than 
text for communicating complex content because processing displays can 
be less demanding than processing text. 

(Vekiri 2002, 262) 


Proponents of this account state that visualisations enhance the process of 
learning at the following levels: 


1 Such representations offer information both about their individual elem- 
ents and their relations (see ibid., 281). Graphical displays make it easier 
for recipients to learn about those elements and they support inferences 
about their relations simply by looking at the depictions. This percep- 
tual feature allows for further merits of visual representations in cognitive 
processes. 

2 In particular, it provides for “computational advantages” (ibid.). That is, 
recipients do not have to search a body of text for the relevant informa- 
tion “and then store it in working memory while searching for the next 
relevant piece” (ibid., 282) — a process that is “prone to error because 
working memory has limited capacity and cannot maintain data for a 
long time without constant attention” (ibid.), as Vekiri points out. Visual 
representations allow this information to be externalised while keeping it 
constantly present before the reasoner’s eyes. In this way, not only is the 
cognitive load reduced and capacities thus saved for further reasoning, 
but also the likelihood of errors is diminished. 
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3 Relying on their perceptual capabilities in deciphering visually presented 
information can enable recipients to draw inferences almost automat- 
ically about the information offered, instead of involving them in long 
interpretive activities (see ibid., 282). Just by looking at a bar graph, the 
student may simply see a difference in length expressing a difference in 
quantity. 

4 Visual representations can support the recipients’ cognitive processes by 
providing them with a concrete mental image that can help to work out 
the solution to a problem in the following way: “[w]hen people reason 
about a problem using symbolic representations they do not have to men- 
tally carry out all the thinking processes but, instead, they can think of a 
solution by manipulating parts of visual images. Reasoning often requires 
consideration and evaluation of alternative possibilities” (ibid.).> 

5 Finally, and linking the current discussion to what has been previously 
said about dual coding, presenting information visually “may trigger the 
recall of relevant knowledge” needed for ongoing reasoning processes 
(ibid., 283). 


This last point, as well as the penultimate one, do not seem to restrict them- 
selves to graphical displays but exhibit an advantage, rather, in the processes 
of reasoning facilitated by all different kinds of visual representations. I would 
therefore claim a broader applicability for the visual argument hypothesis than 
Vekiri does in her article. However, care should be taken not to overempha- 
sise the advantages mentioned, as they are dependent on at least two further 
conditions. 

Firstly, visual representations can be more or less apt for particular 
communicative purposes. Quite a few scholars discuss how to improve the 
design of visual representations to have them result in a higher efficiency. 
Vekiri mentions design guidelines for graphical displays (see ibid., 301ff.). 
Suggestions with respect to graph design are, for example, put forward by 
Priti Shah and James Hoeffner (see Shah and Hoeffner 2002, 62f.). Alexander 
Renkl and Katharina Scheiter discuss proposals of design enhancements on a 
more general level (see Renkl and Scheiter 2015, online first). 

Secondly, learner characteristics play a significant role in creating the 
epistemic advantages of visual representations in cognitive processes (see 
Schnotz 2002, 113f.). Although a variety of aspects are discussed in this 
regard,“ the most significant factors seem to be related to the following abil- 
ities, highlighted by Schnotz: 


[v]isuo-spatial text adjuncts and other forms of visual displays can 
support communication, thinking, and learning only if they interact 
appropriately with the individual’s cognitive system. Accordingly, the 
effects of visuo-spatial adjunct aids depend on prior knowledge, cognitive 
abilities, and learning skills. 

(Schnotz 2002, 113, my italics) 
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The topic of prior knowledge will be briefly examined at this point. 

Apparently this category contains two different kinds of knowledge, one 
related to the depictive style of the visual representation presented to the 
learner and another connected to the informational content of the image at 
hand. The former aspect seems to be naturally relevant, as beyond naturalistic 
depictions, ways of visualising information have been increasingly developed. 
This developmental process has been accelerated significantly by the possibil- 
ities offered by IT devices within the last few decades. Thus, it comes as no 
surprise that people have to learn new depictive styles to correctly decipher 
information presented with their aid. However, this does not undermine the 
previous thesis that we can nonetheless draw on our perceptual abilities and, 
therefore, on evolutionarily manifested advantages of processing visual infor- 
mation when decoding visual representations. For instance, pattern detec- 
tion can be made to work even though we are not familiar with a particular 
style of depiction. Thus, what we are facing here seems to be similar to what 
happens in the course of concept acquisition: we can receive information via 
our perceptual apparatus even if we do not possess the correct concepts of 
what we perceive. Yet our abilities as observers can be improved significantly 
by acquiring the relevant concepts. 

The second, content-related aspect of background knowledge seems to 
be more controversial in the debate. On the one hand, scholars suggest that 
more background knowledge in the related domain enhances the cognitive 
efficiency of visual representations used for purposes of information trans- 
mission (see Vekiri 2002, 304).5 On the other hand, it is pointed out that 
more background knowledge might diminish the cognitive value of visual 
representations as recipients can, for example, visualise relevant details by 
reading a text alone (see e.g. Schnotz 2002, 114). Moreover, it is argued that 
people equipped with a higher degree of background knowledge might also 
be more tempted to disregard visual representations as relevant sources of 
new information and simply experience them as an entertaining side-effect. 
Renkl and Scheiter discuss this learners’ bias, that is, the tendency to ignore 
information presented only visually, as one of the main problems affecting 
the use of visual representations in educational environments (see Renkl and 
Scheiter 2015, online first). 

It seems reasonable to assume that a higher degree of background know- 
ledge can cause both effects, namely a better understanding and a tendency to 
neglect information presented in a visual way, as these are no contradictory 
effects per se. Only their contingent combination will, without doubt, under- 
mine the positive effect that images might have in this context. Renkl and 
Scheiter stress a point important in the context of education. Apparently 
students have to be instructed to acknowledge the relevance of visually 
presented information correctly. However, the neglect mentioned here does 
not extend more generally to the context of scientific discourse. Scientists 
publishing and reading articles usually acknowledge the relevance of visual 
representations, as illustrated above, by drawing attention to the invention 
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of the database INSPIRE in the natural sciences that allows the storage and 
search for visual information separately (see www.projecthepinspire.net, 
accessed February 16, 2016). 

Beyond that, scholars in educational psychology suggest that presenting 
information visually is particularly helpful to students with low prior domain 
knowledge. As Schnotz states: “[p]revious research has pointed out that 
comprehension among learners with low domain knowledge (but sufficient 
visuo-spatial cognitive skills) is increased when pictures are added to a text” 
(Schnotz 2002, 114). Again, this seems to be a reasonable claim, as visual 
representations can guide the learner’s attention to notice the relevant details, 
can highlight relations otherwise overlooked, or present complex informa- 
tion in a significantly simplified fashion. To be concise, students’ prior know- 
ledge seems to be a somewhat ambiguous condition influencing the process of 
learning by using visual means. 

Summing up the previous discussion, it can be stated that, after recipients 
have mastered the initial obstacles to work with visual representations effect- 
ively (which is often part of their scientific training), the latter can, according 
to the visual argument hypothesis, support the cognitive process of learning 
on at least three different levels: firstly, by showing the relation between indi- 
vidual pieces of information; secondly, by making information directly per- 
ceptually accessible; and, thirdly, by enabling a more efficient use of cognitive 
resources. This last aspect is highlighted by Vekiri. “Also, displays support 
thinking during problem solving because they reduce the amount of infor- 
mation that must be maintained in working memory” (Vekiri 2002, 288). 
Moreover, that visual representations indeed bring about these theoretically 
proclaimed advantages is demonstrated by several empirical studies cited by 
Vekiri and others. 

Provided with this empirical background information concerning the cog- 
nitive effectiveness of visual representations, the epistemological analysis can 
now be continued. What exactly the contributions of images might be to the 
two components of cognitive processes such as learning, namely (1) to acquire 
knowledge and (2) to achieve understanding will be examined in the following 
two sections. 


4.3.2 Visual representations and the varieties of knowledge 


Concerning the epistemic functions of visual representations, the discussion 
hitherto has been about the possibility of gaining propositional knowledge via 
perceptually deciphering the encoded information. By analogical reasoning, 
my suggestion was that if it is an acceptable thesis that propositional know- 
ledge can be gained via perception, then the same should be allowed for pic- 
ture perception. This general statement was then limited by the addition of 
the following constraints. 

Firstly, there are kinds of visual representations in science that presup- 
pose more background knowledge than others in order to be interpreted 
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correctly. The ability to decipher information presented in diagrams and 
graphs, for example, usually presupposes a certain training. In a similar 
fashion, Fleck has emphasised that observational skills in science relying 
on the use of instruments have to be learnt, that is, the students have to 
be trained how to ‘see’ correctly. In particular, he shows how a lack of 
training might contribute to misinterpretations of results in microscopy 
(see Fleck 1986b, 118ff.). In such instances, our perceptual abilities are 
not sufficient to gather all of the relevant information, as either the phe- 
nomenon or the form of its presentation does not belong to what we are 
evolutionarily familiar with to observe.5 Here, testimony and percep- 
tion are equally relevant to gather the information presented by those 
visual means. Such an interplay between different epistemic sources is 
not unusual — on the contrary, it seems to be common practice, as Scholz 
explains (Scholz 2009c). 

The second constraint pointed out is closely connected to this co-operative 
outcome of epistemic sources. It seems that our perceptual apparatus yields 
propositional knowledge if we have learnt the relevant concepts beforehand. 
Thus, although our perceptual apparatus often allows us to navigate in the 
world without difficulty, we often need additional explanations to categorise 
phenomena correctly — that is, we need the relevant concepts that are often 
transferred via a combination of showing and telling, via perception and tes- 
timony. In the same way, a correct interpretation of certain images might pre- 
suppose the prior acquisition of relevant concepts. 

Promising as the suggested capability to transmit propositional know- 
ledge already sounds for the possible epistemic prospects to expect from the 
usage of visual representations in science, I wish nevertheless to examine 
another epistemic dimension of scientific images in this section. This add- 
itional epistemic potential of visual representations draws on two aspects 
discussed above, namely on the theory of dual coding and on the capacity to 
transmit non-conceptual content. The first point, put forward and empiric- 
ally defended in the cognitive sciences, refers to the fact that the human brain 
can apparently store incoming information both visually and propositionally. 
Combining this with the thesis that picture perception can also transmit non- 
conceptual content, the theory of dual coding offers an explanation for why 
presenting information visually in the scientific discourse can constitute a 
proper epistemic merit. By using visual means, we can provide others with 
information, namely non-conceptual, that cannot be transferred otherwise. 
Moreover, as the theory of dual coding shows, our brain is apparently able 
to process visual information separately. Thus we are cognitively able to 
handle this information without translating it into propositionally structured 
expressions. The implications of this line of reasoning for acquiring the 
different types of knowledge analysed in epistemology will be discussed in 
what follows. Although philosophers are mainly concerned with propos- 
itional knowledge, i.e. knowing-that, there is nevertheless a variety of other 
epistemic concepts that has caught their attention. Grundmann mentions the 
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following four kinds of knowledge in his introductory work to epistemology 
(see Grundmann 2008, 86): 


1 propositional knowledge (knowing-that) 

2 knowledge by acquaintance 

3 phenomenal knowledge (knowledge about qualia) 
4 knowing-how (skills) 


Although there is this variety of different kinds of knowledge, Grundmann 
points out that epistemologists are primarily concerned with knowing-that (see 
ibid., 71). Eva-Maria Jung explains what reasons are commonly mentioned 
to justify this prioritisation. She identifies two different arguments put for- 
ward as a rejoinder to the claim that epistemologists wrongly focus on propos- 
itional knowledge (see Jung 2012, 13). Firstly, philosophers claim a difference 
in essence between knowing-that and knowing-how. And, as epistemology is 
exclusively concerned with propositional knowledge, knowing-how simply 
does not belong in the scope of its analysis. Secondly, knowing-how can be 
reduced to knowing-that. The former therefore need not be considered as an 
independent kind of knowledge. 

Grundmann analyses different arguments to reduce knowledge of the 
kinds (2) to (4) of the list above to knowing-that (see Grundmann 2008, 74ff.). 
He comes to the cautiously formulated conclusion that it might be possible 
to reduce all of them to propositional knowledge (see ibid., 85). Moreover, he 
claims that even if such a reduction might not be possible, there are two good 
reasons” supporting the prioritisation of propositional knowledge in epis- 
temology: (a) only propositional knowledge can be communicated and thus 
shared within a community; moreover, only propositional knowledge can be 
cognitively processed further via valid inferences, and (b) only propositional 
knowledge is in line with the aim of truth in epistemology (see ibid., 86). 

Now, the above analysis provides argumentative means to broaden the 
focus of epistemology, as it permits calling into question at least one of the 
two reasons mentioned by Grundmann. It will be seen in due course that 
the claim about communicability can be easily rejected. In the same way, the 
results we hitherto obtained can be used to show that there are no particular 
difficulties in cognitively processing visual information. 

In the following discussion, the focus will be primarily on knowing-how. This 
more detailed analysis will allow a brief comment on knowledge by acquaint- 
ance and phenomenal knowledge. All three concepts are related to learning and 
the sharing of knowledge more broadly in communicative contexts. However, 
it can be noted that, whereas knowing-how seems to be of special relevance 
for educational purposes, in science, and thus in relations between experts and 
laypeople, phenomenal knowledge might be of relevance to experts per se. If 
it can be shown that at least one of the kinds of knowledge mentioned above 
can be promoted via visual representations in science, their epistemic rele- 
vance — in comparison to linguistic representations — seems then to be proven. 
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The topic of Anowing-how as an independent epistemic category, i.e. inde- 
pendent of propositional knowledge, is discussed broadly both in epistem- 
ology and in the philosophy of mind. To explain this concept, I will follow 
Eva-Maria Jung and Albert Newen’s suggestion to draw a distinction between 
theoretical and practical knowledge (see Jung and Newen 2011, 95). This dis- 
tinction is explained as follows: 


[t]heoretical knowledge [...] describes a relation between a subject and a 
proposition thereby being related to a norm of truth. [...] Practical know- 
ledge, instead, describes a relation between a subject and an activity. This 
knowledge is related to the norm of success: We ascribe some ability to a 
person if she is able to successfully perform it. 

(Jung and Newen 2011, 95) 


In addition to this, Jung and Newen point out that practical knowledge also 
implies a kind of warrant concerning the ability to perform the relevant action. 
This supplementary condition is necessary to exclude cases of performing 
some action x to bring about y that happen to be successful by mere chance. It 
would be counterintuitive to call such instances ‘knowledge’ (see ibid.). 

According to this distinction, knowing-how belongs to the category of 
practical knowledge. As was highlighted in Grundmann’s list of knowledge 
categories above, knowing-how is commonly regarded as consisting in cer- 
tain skills. Knowing how to ride a bicycle or how to play the piano are 
common examples in philosophy. In science, we might think of instances 
such as knowing how to set up certain experiments, how to use instruments 
such as microscopes, or how to write a scientific article, etc. Including 
all kinds of accidentally successful action performances as instances of 
knowledge-how is inappropriate, as is subsuming all kinds of reflexes under 
this label. Therefore, Jung suggests as a criterion of demarcation that only 
actions brought about intentionally belong to the domain of practical 
knowledge. These intentional acts can be further characterised either by 
being directed at a particular aim or by certain formal aspects of how to 
perform the respective action (that is, a conformity to certain rules) (see 
Jung 2012, 158). Moreover, she claims that these intentional actions can 
be influenced by processes of learning and modification by the respective 
subject (see ibid., 159). If the student realises that a certain action does not 
lead to the intended aim, or only via a variety of unnecessary detours, she 
can learn to improve her actions (see ibid.). 

Now, this characterisation of the object of knowing-how makes clear why 
the question dominating the respective debate in epistemology is about a 
possible reduction of knowing-how to knowing-that. If there are rules, say, 
about how to play the piano correctly, then why not argue that the relevant 
knowledge simply consists in knowing that the piano is played in accordance 
with these rules? In particular, philosophers convinced that propositional 
knowledge is the only category relevant to epistemological discussion suggest 
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different strategies, for example, like the one drawing on rule-following just 
mentioned, to reduce knowing-how to knowing-that.°* As Jung points out 
(see ibid., 13), these attempts enable them to maintain their project to exclu- 
sively analyse propositional knowledge without being forced to ignore the epi- 
stemic phenomenon of knowing-how. 

Tempting as this strategy might seem, Jung also shows that there are at 
least two major difficulties that proponents of such reductive approaches have 
to face. On the one hand, they have to explain that, although an epistemic 
subject might know all the relevant rules about how to perform action x cor- 
rectly, she is still not able to do so (the so-called ‘knowledge-action-gap’). 
On the other hand, they have to account for the fact that there are actions 
which we perform to reach a goal that are not guided by rules (see ibid., 48). 
Jung takes these findings as important hints that although “practical know- 
ledge might involve the knowledge of regulative propositions concerning the 
action”, a complete reduction of knowing-how to propositional knowledge is 
not possible (Jung and Newen 2010, 124). 

Contrary to such reductive approaches, she emphasises the relevance 
of knowing-how as an object of inquiry in epistemology in addition to 
propositional knowledge (see Jung 2012, ch. 1.5). She supports this ini- 
tial conviction with further arguments bolstering the dichotomy between 
propositional and practical knowledge. Both with regard to contents trans- 
mitted and to the aims pursued by their means, the two kinds of know- 
ledge differ essentially (see ibid., ch. 3.3). The following list summarises her 
theses on the topic. 


e Content: the object of knowing-how are intentional actions. This kind of 
knowledge is always related to certain contexts and epistemic subjects. In 
this sense, the contents of knowing-how cannot be objectified completely, 
contrary to propositional knowledge, whose content is thus expressible 
by propositional means. 

e Aim: the aim of knowing-how consists in successful action performances. 
Consequently, it can be assumed that the acquisition and deployment of 
knowing-how to perform such actions are guided by a particular norm, 
namely that of successful action performance (contrary to the norm of 
truth in the case of propositional knowledge). 


I agree with Jung that knowing-how and knowing-that should be regarded 
as distinct categories in epistemology. Despite my general sympathy with her 
approach, two critical remarks have to be added about her criteria to char- 
acterise practical knowledge. Firstly, her thesis that the content of knowing- 
how cannot be objectified, i.e. that it necessarily contains subjective and 
private elements, does not seem to be convincing. In particular, there arises 
a certain tension to another claim of hers, namely that knowing-how can be 
taught, particularly by showing how to practice certain actions, and can thus 
be acquired in educational processes (see ibid., 72). If, however, teaching is 
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possible in such a visual format — this issue will be returned to in due course — 
there have to be at least some paradigmatic instances of the action in question 
that can be demonstrated in order for it to be copied by the student. A partial 
objectification thus seems to be possible. 

Secondly, although it seems to make sense to distinguish between prop- 
ositional and practical knowledge along the lines of different norms, there 
remain some doubts about whether she has chosen the correct ones for her 
contrastive project. Especially regarding propositional knowledge, the debates 
in epistemology show that there is no consensus amongst philosophers 
concerning the status of truth. An example: Alvin I. Goldman, who proposes 
a “veritistic approach” in epistemology, that is, who emphasises the relevance 
of true belief as the predominant aim in our knowledge-seeking enterprises 
(see Goldman 1999, ch. 3), has been constantly criticised by others who object 
that he is wrong to put such a stress on truth alone. A similar objection might 
be raised to Jung’s account if she claims truth to be the decisive criterion on 
the part of propositional knowledge in order to enable the relevant distinction 
between both knowledge categories. 

Despite these critical remarks, however, Jung’s argumentation seems to be 
quite convincing. Especially, the fact that her work is located at the interface 
between epistemology and the philosophy of mind allows her to elaborate 
another interesting thesis. In an earlier article, Jung and Newen had already 
pointed out that the constant misunderstandings of Gilbert Ryle’s concept 
of knowing-how consist partly in neglecting the fact that his approach not 
only aims at a semantic analysis of the term, but also poses the question of 
“whether all mental cognitive processes can be analyzed in terms of propos- 
itional knowledge” (Jung and Newen 2011, 84f.). They argue that Ryle’s pro- 
ject, which is embedded in the philosophy of mind, is also meant to tackle the 
topic of how knowledge is represented in the mind so that the latter can pro- 
cess it. Jung and Newen elaborate on this analysis of representational modes 
of knowledge in their own hypotheses about the distinction between propos- 
itional and practical knowledge. 

They suggest three different modes of representation, namely “(i) propos- 
itional representations, (ii) sensorimotor representations and (iii) image-like 
representations” (ibid., 96). They characterise the first category as “language- 
like” (ibid.). Jung specifies this later as implying the ability to bear truth values 
and as being conceptual in kind (see Jung 2012, 164). 

Contrary to this, the second category is considered as being non-conceptual 
in kind. Moreover, as Jung and Newen point out, the latter is also closely 
connected with certain qualities of our environment that we perceive and 
which trigger certain actions (see Jung and Newen 2011, 97). These repre- 
sentational means are deeply intertwined with our perceptual abilities. Jung 
explains that the human body is the object of sensorimotor representations. 
What is represented in the mind about a particular action are its expected 
duration, the kind and quantity of bodily forces to perform this action and 
certain motoric rules (see Jung 2012, 171). Although practical knowledge 
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can entail rules that are expressible by linguistic means, and although the 
respective skills might at least partly be acquired by such propositional know- 
ledge, Jung defends the claim that practical knowledge is usually represented 
in the sensorimotor way. She presents two aspects to support this claim: on 
the one hand, if the performance of certain skills is interrupted or disabled, 
for example because of certain diseases, we will note that we will not have 
recourse to propositional knowledge for guidance. On the contrary, attempts 
to consciously focus on the action in question will usually be experienced as 
disruptive rather than as helpful. On the other hand, Jung claims that small 
children and animals who do not possess a language are able to perform rea- 
sonable actions nonetheless and can thus be said to possess practical know- 
ledge of certain kinds (see ibid., 177). Of course, assuming that knowing-how 
and knowing-that are represented differently in the human mind also lends 
further support to Jung’s thesis that knowing-how and knowing-that are two 
independent epistemic categories. 

Finally, Jung and Newen suggest a third category of mental 
representations, which they call “image-like”. Contrary to sensorimotor 
representations, they are independent of concrete situations, as they can 
also be triggered by our imagination. Furthermore, they are “systematic- 
ally connected with perceptual images and sensorimotor representations” 
and “with other image-like patterns” (Jung and Newen 2011, 98). The 
interesting thing about this third category that Jung highlights is the possi- 
bility of making use of image-like representations in educational contexts. 
She points out that, whereas sensorimotor representations cannot be con- 
sciously accessed, image-like ones can, and are thus employable as a medium 
to transfer the relevant knowledge for educational purposes (see Jung 2012, 
180). To begin again with a critical comment on this: it does not quite con- 
vince that we cannot consciously access the sensorimotor representations 
of certain actions. From my point of view, it is exactly this that happens if 
people are asked to show or demonstrate the performance of certain skills. 
Admittedly, they will not consciously process all the details of this action, 
but they can call to mind the way they usually perform it in order to dem- 
onstrate it. This seems to suggest that they are at least aware of what is 
essential to this action to be demonstrated so that their students can copy 
them. Again, this line of reasoning is already implied in Jung’s own work 
when she discusses Edward Craig’s pragmatistic conception of knowledge. 
In this context, she highlights the fact that Craig’s concept of the good 
informant, which Craig takes to lie at the heart of our ordinary concept 
of knowledge, not only entails classical testifiers but also people who can 
answer the question they were asked not by telling, but only by showing 
how to do something (see Jung 2012, 72). 

Despite this critical remark, Jung makes a good point in arguing that 
image-like representations are particularly useful for educational purposes. 
Different components, already discussed above, play a role in this and now 
require piecing together. 
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A start can be made with what seems to be most beneficial when using 
visual representations for educational purposes: it is not only that we are able 
to make use of images to transfer practical knowledge at all — which is not 
possible with the aid of linguistic representations. It is apparently also the 
case that, from a methodological point of view, such an educational prac- 
tice permits a considerable expansion in the size of the audience. Whereas a 
teacher can address only a limited number of students by a direct demonstra- 
tion of a particular action, showing how to do x via images not only allows a 
broader audience to be reached on a synchronic level — for example, by live- 
streaming a lecture — but also diachronically, for example by means of images 
showing how to set up a particular experiment in a textbook. 

After practising with the aid of such images, the students can make use of 
them for mental training, as Jung explains (see ibid., 181). ‘Mental training’ 
means that people are able to imagine certain actions and perform them men- 
tally so that their actual performance of these actions will be later enhanced. 

From an epistemological point of view, these features constitute proper 
epistemic merits. As explained above, knowing-how cannot be translated into 
linguistic expressions, that is, traditional methods of teaching are not possible. 
Showing people directly how to do x might be a way out of this dilemma, 
but it considerably restricts the number of people that can be addressed 
by this demonstration. A methodology of utilising images in this way not 
only permits the transfer of practical knowledge to be made public, but also 
enables teachers to professionalise it. As images, moving or non-moving, can 
be recorded and stored, these educational means are not only reusable in a 
variety of instances, they can also be subjected to performance ratings which 
might suggest modifications to the initial images in order to better meet the 
learners’ requirements. 

Of course, the issue discussed here has already acquired the utmost import- 
ance for people concerned with electronic learning and virtual reality. Max 
Hoffmann and his colleagues, for example, propose utilising virtual-reality 
tools in engineering studies (see Hoffmann et al. 2015). They claim that in this 
way more students can be given practical training, even though their univer- 
sities might not be able to offer them the relevant training in a real laboratory 
because they lack the financial resources to do so. Thus, Hoffmann et al. point 
out another dimension in which to broaden the scope of practical training, 
namely via virtual-reality tools. 

Finally, the whole process can also be reversed, that is, visual representations 
can be used to let experts learn about the implicit knowledge of practitioners. 
That such a kind of knowledge is often present and can play significant roles 
in cognitive processes has been pointed out by Eugene S. Ferguson who 
explains, for example, how non-conceptual thinking has guided the design 
and development of machines by craftsmen and designers (see Ferguson 
1977). Now, as Hoffmann indicates,” by experiencing virtual-reality 
simulations, craftsmen or technicians might then be able to indicate why they 
think that certain newly developed machines do not fit their requirements, 
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despite their lacking the conceptual ability to communicate this linguistic- 
ally to engineers. The reasons that these epistemic merits are possible need 
recapitulating. Two aspects seem to be relevant: firstly, visual representations 
are cognitively accessed via perception. This enables the transmission of non- 
conceptual content. Secondly, Jung’s discussion of different ways to represent 
knowledge in the human mind also makes plain why we can process informa- 
tion presented visually quite easily and without the necessity of translation. 
It simply matches the way we think about certain aspects of the world. Here 
we can link our considerations to some results from the cognitive sciences, 
namely to the theory of dual coding — a connection that is also suggested by 
Jung (see Jung 2012, 178). Proponents of this theory suggest that information 
is encoded in two different ways in the human mind, namely propositionally 
and visually. This hypothesis seems to be supported by empirical studies on 
certain brain lesions that disable one of the two possible ways to store infor- 
mation. An example are patients suffering from aphasia, that is, the loss of 
their ability to communicate linguistically, who might nonetheless be able to 
communicate by visual means (see Sacks 2010, 45f.). 

To summarise the results so far, visual representations can play an essen- 
tial, even indispensable part in educational processes focusing on knowing- 
how. Consequently, we are entitled to maintain that some scientific images 
possess an epistemic status that is independent of other representational 
means and thus not epistemically reducible to them. Having discussed visual 
representations in the context of propositional and practical knowledge, 
another of their epistemic merits will now be examined that is connected with 
categories two and three of Grundmann’s list above: knowledge by acquaint- 
ance and phenomenal knowledge. 

What I am about to suggest is best grasped by starting with a negative 
point. It was explained above that the dual coding theory suggests two 
different ways of mentally processing and storing information: one dealing 
with propositional, the other one with visually presented information. 
Apparently the student’s mind works best if both ways are activated during 
the process of learning (see Eitel and Scheiter 2015, 153). As Alexander Eitel 
and Katharina Scheiter argue from an educational-psychology point of view, 
this effect — also known as the “multimedia effect” — obtains not only when 
text and image are presented simultaneously to the student, but also when 
presented sequentially (see ibid., 154). However, what happens if one of these 
ways is completely omitted? More specifically, what happens if the student 
in question only receives propositional information about subject x and no 
visual clues, or vice versa, only visual ones but no propositional information? 
Does she nonetheless acquire a kind of knowledge in either of these cases? 
Regarding this question, it seems relevant to consider two aspects, namely the 
background knowledge of the particular student and her preferred style of 
learning. The latter refers to the fact that some people are more apt to process 
visual information than others, who prefer linguistic explanations (see e.g. 
Kirby, Moore and Schofield 1988). 
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To start with the presentation of mere visual information, it can be assumed 
that people with less background knowledge will have greater difficulty cog- 
nitively processing the information presented, and thus understanding what 
they are supposed to learn. A reference can be made here to what has already 
been said about image games: the student might be able to decipher what is 
shown in the image, but she might not at all understand what her teacher, 
by presenting this information, intends to tell her. As Kjorup has pointed 
out, most images are in need of linguistic anchoring in order to be useful 
in a communicative situation — such as in educational processes. The second 
case, that is the presentation of mere linguistic information, is illustrated by 
Wartenberg’s example of guidebooks for bird-watchers. Suppose, for the sake 
of argument, that such guidebooks do not contain images but only linguistic 
descriptions and explanations. What kind of information will be lost in this 
scenario? What might be learnt from images, but not from the corresponding 
text? Suppose that you opened this somewhat informationally impoverished 
book on the page describing wrens. Suppose further that you have never seen 
a wren — neither in the wild or depicted in any way — before reading this entry. 
Now consider the following two questions: do you think that after reading the 
entry you would be able to recognise a wren if you saw it in the wild? Could 
you Imagine its appearance? It can be assumed that you would not be able to 
do either of these. Of course, your background knowledge about other birds 
might help to exclude some completely odd mental images of a wren. For 
example, the entry tells you that it is smaller than the common house sparrow. 
Thus, it would be rather unlikely that you would imagine a bird the size of a 
dove after reading it. Nonetheless, the text alone will not provide you with the 
kind of information necessary to create a mental image of this bird. Images, 
on the other hand, can easily provide us with this kind of information. That 
is, they can acquaint us with physical entities of which we have no genuine 
experiences such as those gained by watching them in the wild or in a zoo. 

This example thereby brings together aspects from both categories of 
knowledge, that is, knowledge by acquaintance and phenomenal knowledge, 
without completely agreeing with either so that we could adopt the respective 
label for the current case. Let me explain. 

On the one hand, the above example apparently has a lot in common with 
Frank Jackson’s famous Mary argument in the philosophy of mind. It reads 
as follows: 


Mary is confined to a black-and-white room, is educated through black- 
and-white books and through lectures relayed on black-and-white televi- 
sion. In this way she learns everything there is to know about the physical 
nature of the world. She knows all the physical facts about us and our 
environment, in a wide sense of ‘physical’ [...]. If physicalism is true, she 
knows all there is to know. For to suppose otherwise is to suppose that 
there is more to know than every physical fact, and that is just what phys- 
icalism denies. [...] It seems, however, that Mary does not know all there 
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is to know. For when she is let out of the black-and-white room or given 

a color television, she will learn what it is like to see something red, say. 

This is rightly described as /earning [...]. Hence, physicalism is false. 
(Jackson 1986, 291, his italics) 


There are similarities as well as dissimilarities to the bird-guide example. 
What is similar is the way Mary and the prospective bird-watcher learn about 
their subject matter first. Both of them get cognitive access in a somewhat 
limited way only. Mary has to learn about colours without ever experiencing 
one and, likewise, the prospective bird-watcher has to learn about wrens. Both 
of them lack certain qualitative information in what they learn about their 
subject matter. The difference between both examples is, of course, the object 
of learning. Jackson’s argument is about qualia — i.e. the question what it is 
like to experience something, say, red, whereas the bird-watcher example is 
about physical objects in general. 

I do not want to dwell on Jackson’s argument, as my point is not about 
qualia and physicalism in the philosophy of mind.‘' Despite the difference 
in direction, Jackson’s argument nevertheless makes it very plausible that, 
in some instances, linguistic descriptions alone will not suffice to provide us 
with the necessary information to construe a correct mental representation 
of the entity in question — neither of a certain colour nor of the appearance 
of a particular bird.” Images, however, often allow us to construct a mental 
representation without great effort. Such mental images can then be used, 
for instance, to recognise examples of the same species in the wild by com- 
paring the mental image and the visual appearance of the bird in question. 
Yet because of the directional difference between Jackson’s argument and the 
bird-watcher example, I am reluctant to call what has been learnt by the stu- 
dent about the visual appearance of the wren phenomenal knowledge. 

Similar difficulties arise with regard to the second category, namely know- 
ledge by acquaintance. Nonetheless, it can be assumed that it describes very 
well what happens cognitively when dealing with instances like the two 
examples just mentioned. Bertrand Russell, who introduced this concept in 
epistemology and compares it to what he calls “knowledge by description”, 
defines it in the following way: “I say that I am acquainted with an object 
when I have a direct cognitive relation to that object, i.e. when I am directly 
aware of the object itself” (Russell 1910-1911, 108). Obviously, such a direct 
cognitive relation to the object in question is what is missing in the case of 
Mary and the prospective bird-watcher above. Neither of them has direct 
access to their object of interest, namely colours or wrens. This relational 
character of ‘acquaintance’, that is, the relation between the epistemic subject 
and the object in question, is particularly stressed by Russell as one of its con- 
stitutive characteristics (see ibid., 109). 

Moreover, as Ali Hasan and Richard Fumerton point out, knowledge 
by acquaintance “is knowledge of something and logically independent of 
knowledge that something is so-and-so” (Hasan and Fumerton 2014, sect. 
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1). Summarising Russell’s account, they add that “for Russell acquaintance 
is nonjudgemental or nonpropositional” (ibid.). These are the aspects of 
knowledge by acquaintance that shall be emphasised here. Apparently this 
conceptual framework fits well with what has hitherto been stated about the 
acquisition of non-conceptual content via perception — either in a direct 
way or mediated via visual representations — and its processing and storage 
in a separate cognitive subsystem, as indicated by proponents of the theory 
of dual coding. 

Unfortunately, just as I cannot adopt the concept of phenomenal know- 
ledge for my purposes, neither can I wholeheartedly subscribe to the theory 
of knowledge by acquaintance. The dispute is about the objects of these 
knowledge-relations. Russell suggests that “sense-data” should be regarded as 
the paradigmatic instance of objects of acquaintance (see Russell 1910-1911, 
109). He explains: “[w]e shall say that we have acquaintance with anything of 
which we are directly aware, without the intermediary of any process of infer- 
ence or any knowledge of truths” (Russell 1912, ch. 5). 

Yet I am reluctant to commit my approach to the sense-datum theory 
because this account entails many well-known problems. According to 
this theory, for example, sense-data are solely bound to private subjective 
experiences, which leaves unresolved the question of whether we can ever 
know that we are talking about the same objects if they are given to us in this 
private and subjective way (see Grundmann 2008, 475), despite the fact that 
people have no particular difficulty in agreeing about what they perceive, for 
example how many chairs and tables they see in a particular room. Thus, there 
has to be an object of reference that guarantees this inter-subjective agreement 
on the facts. Hence, an account of knowledge by acquaintance that focuses on 
sense-data as an appropriate epistemology cannot be adopted here. 

At first sight this might be a rather unsatisfying result, as neither of the 
two approaches can be accepted as the proper epistemic framework for the 
purpose of explaining what kinds of knowledge are transmitted via images. 
However, neither the theory of phenomenal knowledge nor the account of 
knowledge by acquaintance was designed for this purpose, and consequently 
criticising them for not accounting for this phenomenon is inappropriate. 
However, this discussion was not intended as a critique. On the contrary, 
the above has shown in what sense both accounts might, despite those dif- 
ficulties, contribute to a better theoretical understanding of how it is pos- 
sible to acquire information relevant to constructing a mental image of the 
entity in question. Images can transmit a non-propositional content. In 
some instances, this content entails that all relevant information constructs 
a corresponding mental image. Both theoretical approaches, dealing with 
questions about either phenomenal knowledge and knowledge by acquaint- 
ance, support the thesis that there is more to human cognition than mere 
propositional knowledge. They suggest models for how to comprehend the 
acquisition of non-propositional knowledge. Consequently, the proposal is to 
take those approaches as a starting point to develop a proper epistemological 
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account concerning information distribution and acquisition by means of 
images. To be concise, visual representations are relevant in scientific dis- 
course as they can be used to convey a mental presentation of the appearance 
of the entity depicted. They can, so to speak, foster an inner picture of it by 
transmitting non-conceptual information that is processed in a separate cog- 
nitive subsystem. This information will, however, be lost if scientific images 
are replaced by mere verbal descriptions. 

In order to elucidate the role of imagination in science, Tamar Szabó 
Gendler’s discussion of the intimate connection between imagination and 
counterfactual reasoning should be taken into account (see Gendler 2013, 
ch. 4.4). “It has been argued that imagination plays a central role in figuring 
out what would happen — or what would have happened — had things been 
different from how they in fact are or were” (ibid.). If we can assume that 
there is such a connection, the role of imagination in science becomes obvious 
at once. Counterfactual reasoning is a main feature of the scientific enterprise 
itself. It is needed to invent and perform experiments, to invent theories and 
ways of testing them. Furthermore, it is a basic requirement that natural 
laws permit counterfactual reasoning within their scope, that is, they have 
to remain valid under altered circumstances. Predictions can be given in the 
form of counterfactual statements, for example. In all of these contexts and 
scientific tasks, imagination plays an important role, and so does the know- 
ledge conveyed by visual representations concerning their object of depiction. 

Thus, the capacity to transmit non-conceptual content and to make it 
directly accessible via perception allows us to explain in what sense visual 
representations can contribute to the acquisition of practical knowledge and 
how they can acquaint us with entities so that we acquire mental images of 
them. Beyond that, the fact that we cognitively access visual information by 
perceptual means explains in what sense images can contribute to another 
epistemic desideratum, namely to scientific understanding. This is what we will 
discuss in the next section. 


4.3.3 Visual representations and scientific understanding 


Apart from the acquisition of (propositional) knowledge, learning is 
normally associated with the aim of understanding. Peter Kosso points 
out that solely memorising propositions is not what is expected of our 
students — especially not in science (see Kosso 2007, 175); they are not 
usually required to parrot hypotheses and statements during an exam- 
ination, for example. On the contrary, scientific training ideally means 
having students partake in the community of researchers, that is, enabling 
them to apply acquired knowledge to new questions, to reflect critically 
on this information and, if necessary, to correct some of its components. 
The aspect of understanding now acquires relevance here. Are there any 
particular contributions, then, that visual representations can make with 
respect to scientific understanding? Possible answers to this question will 
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be presented following a brief discourse of this incipient point on the epi- 
stemic relevance of understanding itself. 

Scientific understanding is commonly regarded as an ability to coherently 
fit new items into one’s knowledge system and to apply the newly acquired 
information to solve further tasks and puzzles. Wesley C. Salmon phrases this 
in the following way: “[...] we have scientific understanding of phenomena 
when we can fit them into the general scheme of things, that is, into the scien- 
tific world-picture” (Salmon 1993, 12f.). But how exactly should this fitting- 
relation be conceived? What does Salmon suggest when he claims that ‘to 
understand something’ means ‘being able to fit it into “the general scheme of 
things” ’? An answer to this question is offered by Jonathan L. Kvanvig who 
emphasises that this fitting-relation is the crucial difference between know- 
ledge and understanding. He states: 


[...] that understanding requires, and knowledge does not, an internal 
grasping or appreciation of how the various elements in a body of infor- 
mation are related to each other in terms of explanatory, logical, probabil- 
istic, and other kinds of relations that coherentists have thought constitutive 
of justification. 

(Kvanvig 2003, 192f., my italics) 


In a similar fashion, Kosso suggests that understanding in science goes 
beyond a mere additive compilation of evidence on a certain matter (see Kosso 
2007, 179). That there is something important lacking in cases when eviden- 
tial facts are collected in science without the achievement of understanding is 
highlighted by several examples. For instance, he draws his readers’ attention 
to how the phenomenon of contagion was discovered (see ibid., 182) and 
points out that, although Thucydides in ancient Greece reported in detail how 
people became infected with the plague through nursing patients already sick, 
he did not infer the mechanism of contagion lurking in the background. As 
Kosso puts it: 


[h]is knowledge, however, stops with the isolated fact of the disease 
somehow being transmitted from one person to another. [...] He is not 
credited with the first proposal of the germ theory, since he did not under- 
stand the process of infection. 

(Kosso 2007, 183) 


It is in this sense that Kosso claims that scientific observation alone might 
yield factual knowledge but not understanding (see ibid., 184). Taking the 
above discussion into account, this statement seems surprising — but only at 
first sight. Two clarifying remarks should be added at this juncture. 

Firstly, Kosso’s thesis that observation — respectively (visual and pictorial) 
perception — can yield propositional knowledge if concepts are available is 
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acceptable, and yet (secondly) his statement is still puzzling in that the way he 
phrases his ideas about scientific observation seems to suggest that the latter is 
more or less independent of theoretical considerations which, of course, it is 
not. Elsewhere, he has discussed in detail the intertwining of observation and 
scientific theories (see e.g. Kosso 1988; 1993). Thus, to understand his claim 
correctly, we might interpret him in the following way: observation is theory- 
related in different ways. Yet it might be the case that the theories used are not 
adequately embedded within the broader network of scientific theories that he 
calls attention to. If this happens — either by deliberately screening off other 
theoretical assumptions that might lead to a rejection of the theory at hand 
or unwittingly as a consequence of the observer’s own scientific training, a 
proper understanding of the phenomenon at hand will be blocked as in the 
Thucydides example above. 

However, even though I agree with Kosso’s suggestion, another constraint 
to his thesis should be added, namely that understanding is usually a gradual 
matter. This assumption seems to be perfectly in line with Kosso’s ideas. If 
less background knowledge is available that the scientist can use to interpret 
her observations, her understanding of the phenomenon at hand will also be 
affected in a negative way. It can be stated, then, that Thucydides did under- 
stand that the disease observed was transmitted from person to person, but he 
did not understand how this happened. Consequently, his understanding of 
disease transmission was only partial, and not, as Kosso suggests, completely 
lacking. 

The topic of understanding has rather recently started to attract the 
attention of epistemologists. This new focus of research is an aftermath of 
the challenge posed by the Gettier-cases concerning the analysis of know- 
ledge. One consequence of this discussion has been to bring into focus the epi- 
stemic aims and values a subject might pursue when seeking knowledge (see 
Jung 2012, ch. 2.4.3). It is here that the debate about understanding as an epi- 
stemic desideratum valuable in its own right begins (see Pritchard and Turri 
2014, sect. 5). Moreover, it is also here that we get a clearer grasp of what 
motivates Kosso’s critique on regarding the striving for evidence as the correct 
aim of science. Philosophers concerned with the topic of understanding usu- 
ally point out that truth is but one aim valued in epistemic projects. Thus, 
amassing true beliefs might be a laudable facet of science, but does not reveal 
its whole epistemic enterprise. 

In particular, this debate has been fuelled by Kvanvig’s work (see Kvanvig 
2003). He discusses the question of whether understanding is a species of 
knowledge. Kvanvig starts his analysis by pointing out that it is commonly 
assumed that understanding and knowledge are closely connected (see ibid., 
188). It is usually said that if a student understands that x, she also knows 
that x. In instances like this, the student will possess true beliefs that are 
propositionally structured if she understands the respective information. In 
this sense, propositions can also play a role. This assumed intimate connection 
between knowledge and understanding is now challenged by epistemologists, 
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some of whom even call for a replacement of ‘knowledge’ by ‘understanding’ 
in epistemology as a consequence of this critical discussion.™ 

What is at stake here is the factivity of understanding. What does this 
mean? Obviously, we can have knowledge without understanding (see ibid., 
191), as the above example of the student learning by rote shows. But does 
this claim also hold the other way around, i.e. do we need knowledge as a 
basis for understanding? Usually it is admitted that there are factive and non- 
factive usages of the term ‘understanding’. Factive understanding implies 
truth in the same way that knowledge does, as Kvanvig points out (see ibid., 
190). Non-factive understanding, then, is either due to “misspeaking or to 
the expression of propositions that do not involve the concepts of knowledge 
or understanding central to epistemological inquiry” (ibid.). Statements such 
as, ‘I understand that he was not able to attend the conference’, illustrate this 
latter case. 

However, non-factive understanding is often more or less immediately 
ruled out as a candidate for epistemological investigations, which is Kvanvig’s 
line of argument (see ibid., 190f.). He explicitly draws our attention to factive 
understanding and identifies two kinds, namely: 


propositional understanding and objectual understanding. The propos- 
itional sort occurs when we attribute understanding in the form of a prop- 
ositional operator, as in understanding that something is the case, and the 
objectual sort occurs when understanding grammatically is followed by 
an object [...] 

(Kvanvig 2003, 191) 


Although the second sort is not straightforwardly propositional in kind, he 
also thinks that it is factive, as we have to have true beliefs about the object in 
question in order to be attributed with an understanding of that very object 
(see ibid.). Moreover, Kvanvig argues that other varieties of understanding — 
such as “understanding why, when, where, and what are explicable in terms of 
understanding that something is the case” (ibid., 189). He does not consider 
understanding-how as a relevant concept in epistemology — similar to what 
was noted above about the concept of knowing-how — as it is patently more 
closely related to practical concerns than to theoretical ones (see ibid., 190). 

The feature of factivity seems to suggest that understanding and know- 
ledge are somehow on a par, that is, from an epistemological point of view, 
understanding does not seem to contribute anything new. If I know that Pluto 
has five satellites, I do understand that Pluto has five satellites. Both — knowing 
that p and understanding that p — presuppose that p is true, as Kvanvig makes 
clear. Despite this initial similarity, however, he defends the claim that there is 
also an important distinction, namely: 


[...] once we move past its factivity [of understanding, N.M.], the grasping 
of relations between items of information is central to the nature of 
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understanding. By contrast, when we move past the factivity of know- 
ledge, the central features involve nonaccidential connections between 
mind and world. 

(Kvanvig 2003, 197) 


Thus, from Kvanvig’s point of view, understanding is independently epis- 
temologically valuable because it involves the grasping of coherence relations 
amongst different true beliefs, and this grasping also contributes to the 
systematising and organising of our belief system (see ibid., 202). 

Other philosophers defend conceptions of understanding that diverge 
even more radically from the concept of knowledge than Kvanvig’s 
claims suggest. In particular, many scholars are less convinced of the fac- 
tive status of understanding. Catherine Z. Elgin, for example, discusses 
understanding, amongst other things, in the context of scientific endeavour, 
and points out that it would not make much sense to demand factivity in 
this setting. 


The growth of understanding often involves a trajectory from beliefs that, 
although strictly false, are in the right general neighborhood to beliefs 
that are closer to the truth. The sequence may terminate in true beliefs. 
But even the earlier steps in the sequence should fall within the ambit of 
epistemology. For they are, to an extent — often to a considerable extent — 
cognitively valuable. 

(Elgin 2007, 37) 


Her finding can be read both diachronically and synchronically. 
Diachronically, this affects the phenomenon of progress in science in gen- 
eral. Scientific realists point out that our current scientific theories are at least 
approximately true, and hence that we can think of the history of science 
as a developmental process towards truth. Of course, scientists might have 
deviated somewhat in the past (the phlogiston theory, for example), but none- 
theless our theories and thus our knowledge of the phenomena, and also our 
understanding of them, has more or less constantly increased. Insisting on 
the factivity of understanding, that is, maintaining that understanding that 
p implies that p is true, would then expel such approximations to truth from 
the scope of ‘understanding’ completely, which would be a rather counter- 
intuitive consequence. 

A similar case can be made for the synchronic level in science. What is 
critically discussed here by philosophers is the usage of idealisations in cog- 
nitive processes in science (see e.g. Elgin 2007; Mizrahi 2012). Idealisations 
are, for example, brought about by laboratory conditions relevant to most 
experiments yielding observational data. Moreover, they also obtain due 
to making use of models or ceteris paribus laws in processes of reasoning. 
Therefore, idealisations are often the starting point for cognitive processes in 
science, though they are, strictly speaking, not true. They are simplifications of 
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actual phenomena or processes. Particular aspects are intentionally omitted 
in these cases so that the amount of information is reduced. Can we nonethe- 
less understand the (actual) phenomenon or subject matter in question? Our 
intuitions suggest that at least a partial understanding is possible, but a theory 
demanding the factivity of this cognitive achievement would deny this. 

A corollary of what has been stated so far about the concept of 
understanding and its potential relation to factivity is that either we have to 
admit that images can yield propositional knowledge to be understood in an 
epistemologically relevant sense or that they simply do not belong to the scope 
of ‘understanding’. Again, this seems to be a rather counterintuitive conse- 
quence of such an approach to ‘understanding’. In this context, Elgin is correct 
in reminding us that “[w]e also understand pictures, words, equations, and 
diagrams. Ordinarily these are not isolated accomplishments; they coalesce into 
an understanding of a subject, discipline, or field of study” (Elgin 1993, 14). 
Although content with Elgin’s statement that we use the term ‘understanding’ 
also with respect to those different vehicles of information, I suggest some 
clarifying remarks at this point. Earlier in this analysis, Scholz’s suggestion 
was introduced that there is a variety of levels involved when speaking about 
‘understanding a picture’ (see Scholz 1993). Bearing this in mind, we should 
at least make a distinction between the two levels of understanding implied in 
Elgin’s quotation. On the one hand, we understand the respective vehicle of 
information (which might imply a variety of sub-levels of understanding, as 
pointed out by Scholz) and on the other, we understand the contribution that 
the content of this informational vehicle makes concerning the development 
of a theory or even, as Elgin states, “a subject, discipline, or field of study”. 

Anyway, both dimensions of understanding share the same difficulty if we 
focus on visual representations as the respective informational vehicles and 
nonetheless strive to maintain the claim about the factivity of understanding. 
As Elgin points out, images are not propositionally structured, and thus 
lack the capacity to bear truth values in the traditional Fregean sense (see 
Elgin 1993, 27). This provides an interesting twist to the starting point of 
processes of understanding. In the case of idealisations in science, the ini- 
tial step consists in propositional statements that cannot be called ‘true’ in 
the strict sense, since they entail simplifications, etc. Visual representations, at 
least from the traditional point of view, are neither true nor false — nonethe- 
less, as Elgin pointed out above, they can constitute the starting point for a 
process of understanding. 

The thesis is, then, that by contributing to our understanding of scien- 
tific phenomena, visual representations fulfil an important epistemic task. 
Focusing on ‘understanding’ instead of ‘knowledge’ regarding the epistemic 
capacities of images in science allows us to support the claim that visual 
representations can play crucial roles in cognitive processes in this context by 
another argument. This line of reasoning runs as follows. 

Visual representations are correctly taken as heuristic tools in this context, 
namely in the sense of supporting the cognitive process of learning. Yet, not 
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only do they enable students to acquire propositional knowledge, but they 
also allow them to achieve an understanding of the information presented. 
Acknowledging the fact that understanding is an epistemic desideratum in 
its own right now enables a particular twist in the argumentation: whereas 
pointing to the heuristic function of visualisations usually implies the devalu- 
ation of their epistemic status, we can defend the opposite point of view. Jf 
visual representations can facilitate understanding, and understanding is inde- 
pendently epistemically worthwhile, then it can be stated that images can make 
a substantial epistemic contribution. 

However, this line of reasoning presupposes (1) that understanding is 
epistemically worthwhile in its own right. So far, we have only pointed out 
that it is a distinct epistemic desideratum — but why should we strive for 
understanding? (2) We also have to show how visual representations can 
support scientific understanding. That they apparently do play an important 
role here is empirically supported by different studies carried out by educa- 
tional psychologists (see e.g. Müller et al. 2012; Schnotz 2002; Vekiri 2002). 
Yet the question remains how it works. How exactly do images support scien- 
tific understanding? The remaining part of this section is devoted to answering 
these two questions. 

The above examples of learning processes demonstrate that understanding 
adds an important epistemic dimension to our knowledge-seeking enterprises, 
and thus that epistemologists are well-advised to consider more seriously 
understanding as an epistemic desideratum in its own right. However, I do 
not agree with the broader thesis that the concept of knowledge should be 
replaced by ‘understanding’ in epistemology, because, although usually 
intimately linked, they are nonetheless quite independent of each other. As 
a corollary of this relationship, both concepts are relevant to consider when 
theorising about epistemic achievements, projects and practices. A start can 
thus be made by explaining what exactly understanding can add to the epi- 
stemic project in science. 

Philosophers who consider understanding as relevant to epistemology 
commonly stress its additionally epistemic value which is spelled out in the 
grasping of certain connections in a body of information. Kosso expresses 
this benefit in the following way: “[u]nderstanding reveals the larger landscape 
and includes the ability to apply one idea to other situations without being 
given detailed instructions” (Kosso 2007, 176). 

Here, Kosso points out two particular achievements of understanding: 
firstly, what he calls revealing the larger landscape, that is, understanding 
how bits of information in one’s area of research fit together. For example, 
students in the philosophy of science might learn about Karl Popper’s 
account on falsification first, and afterwards about Kuhn’s theory of scien- 
tific revolutions. Learning in addition that these are successive approaches 
to explain (amongst other things) what demarcates science from pseudo- 
science will allow students to understand the sequential development of 
these theories much better, namely as a consequence of a shift in focus 
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on the topic amongst philosophers of science. The second achievement of 
understanding that Kosso points out is that it enables scientists to apply 
their knowledge to answer new questions in their field of expertise. In this 
sense, understanding is an important goal of scientific education because, 
in the long run, it will enable students to do their own research. 

From Kosso’s perspective, this second epistemic accomplishment of 
understanding in science is a direct consequence of the first, that is, the 
understanding of connections (see ibid., 182). Earlier in this section, we 
mentioned Kvanvig’s theses concerning the different kinds of relations pos- 
sible here. Yet what re/ata do they connect? Here, Kvanvig only speaks vaguely 
about “various elements in a body of information” (Kvanvig 2003, 192). Does 
Kosso offer a more precise account in the context of science? He suggests 
that “[t]he achievement of understanding is in apprehending the connections 
between theories and the global coherence among concepts” (Kosso 2007, 
179). Thus, the re/ata that he points out are, on the one hand, theories and, 
on the other, concepts. 

Admittedly, understanding can be addressed as a particular phenom- 
enon in both instances. However, clarifying the exact nature of the relata 
in question also depends on the degree of understanding that the recipient 
is supposed to acquire. It apparently makes a difference, for example, in the 
context of education whether students are required to understand a par- 
ticular law and its applications or to grasp developments in the theoretical 
descriptions of a phenomenon. In particular, it is not always a set of the- 
ories that is linked in cases where understanding is attributed. Nevertheless, 
it can be taken that understanding is particularly valuable as it is based 
on realising the connections between, for instance, theoretical statements. 
One of the epistemic benefits that such a kind of understanding implies, 
without going into detail, is that it allows scientists to explain new phe- 
nomena by using metaphors® — which is also pointed out by Fleck. He 
explains how the invention of metaphors is made possible by the scientist’s 
simultaneous membership of different thought collectives. More precisely, 
Fleck defends the claim that such a metaphoric reasoning is possible by 
making use of background knowledge taken from popular science (see 
Fleck 1979, 112).°° The scientist is enabled to make this explanatory transi- 
tion just by noticing the connection between two concepts or two theoret- 
ical approaches. Moreover, by introducing metaphors in scientific discourse 
we not only get epistemic access to a phenomenon that previously was not 
conceptualised at all, but it might also become the seed of a completely new 
theory. Here, the epistemic relevance of understanding becomes more than 
obvious. Now, in what sense can we state that visual representations support 
such understanding in science? 

To answer this question it is appropriate to refer back to the results of 
educational psychology which explicitly deal with the contributions of visual 
representations in the context of learning. To summarise the results of the 
above discussion, those approaches suggested the following three ways in 
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which visual representations influence the process of learning and how they 
can thus enhance understanding: 


1 Levels of information: visual representations are not only able to 
transmit information about particular items, but also about relations 
among them. 

2 Visual deciphering: visual information is predominantly grasped by 
making use of our perceptual apparatus. For example, comparative tasks 
amongst visually presented items can be performed in this way without 
the need to engage in long interpretations. Just by looking at a bar graph, 
the student may simply see the difference in length expressing a difference 
in quantity, etc. 

3 Cognitive processing: visual information can not only reduce the cognitive 
load on the working memory, but also allows for a more economic hand- 
ling of our cognitive resources in general. 


In the following, we have to analyse in what sense these epistemic virtues can 
not only support learning in general, but also understanding in science in 
particular. 

The first point seems to be obvious. It was said that scientific understanding 
is about grasping connections — especially relations between concepts, the- 
ories and the like. As Kosso puts it: “[u]nderstanding [...] is entirely a matter 
of fitting into a pattern. Understanding depends on coherence” (Kosso 2007, 
181). The patterns that Kosso mentions here can thereby appear on different 
levels. They can link individual concepts or statements, or individual concepts 
and theories, or theories to theories. Such patterns can support inductive and 
deductive reasoning, as they might, for instance, reveal relations of entailment 
or hierarchy. Moreover, those patterns can also link unknown items to already 
known ones — which seems to be the case that Kosso has in mind when talking 
about scientific understanding. 

Now, visual representations can serve exactly this purpose. As Matthew 
T. McCrudden and David N. Rapp point out with respect to image design for 
educational purposes: “[a]n effective visual display as designed for educational 
purposes has two main functions: (1) to communicate important information 
and (2) to communicate relations about information via spatial arrangements” 
(McCrudden and Rapp 2015, online first). Thus, the obvious part to be played 
by visual representations is to show the connections mentioned by Kosso, i.e. 
to literally visualise them. Tree diagrams are a striking example in this context. 
Students are not only expected to learn something about particular items, but 
also about their relations. Visualisations can highlight such relations in an 
immediate fashion, and thus support the cognitive process of understanding. 

McCrudden and Rapp discuss this contribution of visual representations 
to the epistemic processes of students’ education under the labels of “organ- 
ization” and “integration”. The organisation of information in a given image 
can be accomplished at different levels, and thus allows the learner to draw 
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relevant inferences. The authors mention three kinds in particular, namely 
“temporal inference”, “hierarchical inference” and “relational inference”. 
The sequential depictions in Figure 3.3, for instance, will allow the learner 
to draw temporal inferences about the life cycle of a frog. The tree diagram 
mentioned in the previous example can allow for hierarchical inferences 
between concepts. And the diagrams of the detection of gravitational waves, 
discussed in the introduction to this book (see Figure 1.2), can illustrate the 
last kind of inference. Those diagrams allow the recipient to compare the two 
recorded signals to each other and to the curve theoretically predicted for 
such events. 

In addition to those inferences, mentioned by McCrudden and Rapp, there 
are a variety of spatial relations among objects that a recipient can infer by 
regarding images. The Pioneer plaque (see Figure 2.10) can illustrate this. The 
depiction at the bottom shows the place of the spacecraft’s origin, and thus 
how it is related to our solar system. 

Integration is closely related to organisation. McCrudden and Rapp point 
out that what is implied here is the relation between newly presented items and 
the students’ background knowledge. This seems to be exactly what Kosso 
suggests for the case of understanding, namely to connect theories, i.e. newly 
learnt ones to formerly acquired ones. Now, how can visual representations 
facilitate this epistemic achievement? McCrudden and Rapp explain that 
there are two ways in which integration can take place: an active and a passive 
one. In particular, the second one offers an explanation of how images can 
serve the purpose at hand. They can entail clues that will (or, in the case of 
education, are supposed to) activate the students’ prior knowledge. Active 
integration, on the other hand, is guided by the student’s expectation. That 
is, the learner assumes that the visually presented information is somehow 
related to a field of prior acquired knowledge. 

Although McCrudden and Rapp have a point in highlighting the fact 
that visual representations can contribute both to the organisation and the 
integration of information in the processes of learning, it seems that the 
latter, i.e. supporting integration, is not a special achievement of visual 
representations in particular, but can be gained by other representational 
means as well. 

It might be objected that the epistemic achievements just explained are 
only of a secondary quality, as they are intentionally brought about by the 
teacher who utilises the visual representation and thus presents the infor- 
mation depicted in a way that the above-mentioned advantages of organ- 
isation and integration can be exploited. Visual representations contribute 
to understanding, but only in the context of previously known facts. Hence, 
they cannot make contributions to constituting a genuinely new kind of 
knowledge and understanding, but are only vehicles that pass on known 
information. 

We have to admit that images can be used in this way, which holds, inciden- 
tally, for all other kinds of representational tools as well. However, we do not 


316 The cognitive value 


have to agree with the thesis that this is the only way that images can make 
contributions to epistemic processes and to understanding in particular. The 
rationale for this claim is closely connected to the second aspect concerning 
how images can facilitate learning, namely visual deciphering. This perceptual 
mode of access also enables recipients to make use of correlated skills that 
have developed in the course of evolution. In accordance with this, Zachary 
C. Irving argues for a fundamental role of visual representations concerning 
scientific understanding (see Irving 2011). He discusses the difference between 
visual and numerical representations and highlights the fact that, because of 
the limitations of human cognitive capacities, the former are particularly 
useful for the understanding of large data sets. His primary example concerns 
scatter plots which, according to Irving, are especially useful for detecting 
patterns among the data (see ibid., 780f.). As an example let us take a look 
at the Hertzsprung-Russell diagram (see Figure 4.5), showing the correl- 
ation between the temperature and magnitude of stars. Obviously, by simply 
looking at the plot, we can literally see how magnitudes, temperatures and 
luminosities of stars are related and which of these relations are the most 
common. 

I agree with Irving that some visual representations are especially valu- 
able as they enable pattern detection among data. However, this merit should 
be related to our abilities as visual observers and our resulting cognitive set- 
up, but I am reluctant to discuss this as a kind of cognitive limitation. That 
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Figure 4.5 Hertzsprung-Russell (H-R) diagram. 
Source: NASA/CXC/SAO. 
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human observers are particularly apt to perform this task is, for example, 
suggested by projects such as Galaxy Zoo. Such projects demonstrate, con- 
trary to Irving’s example, that human observers can detect patterns whose 
identification is simply not possible at the level of numerical data. That human 
beings are particularly skilled in the task of pattern detection is undoubtedly 
a consequence of evolutionary processes. Thus, making information available 
in a way that also activates these skills can enhance our understanding by 
connecting the cognitive processing of information to these abilities. 

Consequently, the grasping of connections is not necessarily a result of a 
previously intended design. Contrary to such a sceptical approach, it can be 
assumed that not all relations detected with the aid of visual representations 
are previously known and this is because of the excellence of human obser- 
vers in pattern detection. The capacity also to transmit non-conceptual infor- 
mation via images can contribute to such visual discoveries or detections of 
anomalies. 

The final point concerning the ability of visual representations to enhance 
understanding is related to the aspect of cognitive processing. As McCrudden 
and Rapp explain, “[IJearners have limited processing resources. Of par- 
ticular relevance to visual displays are the resources associated with attention 
and working memory” (see McCrudden and Rapp 2015, online first). What 
is important here is a more economic handling of those cognitive resources 
necessary for processing incoming information. Educational psychologists 
suggest that visualisations can constitute a kind of relief for our cognitive 
system which is achieved as follows. 

On the one hand, visually presented information can guide our attention, 
as McCrudden and Rapp point out. In particular, by making use of ‘signal- 
ling’ techniques, important information can be highlighted in an image and 
the learner’s attention can be directed towards it. The diagram of the yearly 
average number of sunspots is a good example (see Figure 4.6). An arrow has 
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Figure 4.6 Diagram: sunspots. 


Source: NASA. For a colour version of this image see www.nasa.gov/images/content/ 
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been added to indicate the peak of 1928 that researchers considered being 
similar to the emerging cycle of sunspots in 2009. This signal directs the 
viewer’s attention directly to the relevant information. 

On the other hand, and more importantly, visual representations can keep 
information and relations among the data available while we think about 
problem solutions. We do not have to store all the information in our working 
memory during this process.°’ In this sense, visualisations might provide the 
necessary cognitive resources to work out the relevant connections in order 
to fully understand a particular topic. This method of supporting the cog- 
nitive processing of information by visual representations is, for example, 
highlighted by McCrudden and Rapp. They explain that images can, for 
instance, reduce the effort needed to select important information, as they 
include signals which make this information more salient. “Similarly, a dis- 
play improves processing efficiency, when it helps a learner organize important 
information more quickly with the display than in its absence or if the display 
is not designed well (e.g., related ideas are not near one another)” (McCrudden 
and Rapp 2015, online first). All of these aspects of information transmission 
with the aid of visual representations can enhance scientific understanding — 
also without a prior translation into propositional statements. The discus- 
sion above has made clear how images can facilitate scientific understanding. 
Our analysis has made plain that images often provide more efficient ways 
to achieve understanding than other representational means, for example, by 
visually organising the relevant information and by literally keeping it before 
the recipients’ eyes. However, although our previous results do support the 
thesis that visual representations can play positive epistemic roles by facili- 
tating scientific understanding, they leave open the question of whether there 
are instances that inevitably call for the utilisation of images to allow students 
to understand what has been presented. Or to put it differently, do we have 
to admit that there are always other ways to achieve scientific understanding 
that work equally well? 

There are different answers to this question. Firstly, as has been pointed 
out by educational psychologists, there are different types of students. Some 
prefer verbal descriptions, whereas others rely on visual representations. 
Thus, a moderate thesis would be to claim that the latter type of learners are 
somehow in need of images to get a clear grasp of what they are supposed 
to learn. 

The phenomenon of visual thinking has been discussed in section 3.1.1 
and the suggestion made that images can play essential roles as tools of 
thinking in the exploratory context of science. As an example, Dr John 
Snow’s discovery of cholera transmission routes was analysed. This dis- 
covery was made possible by his working with different maps arranging 
various facts that could offer a possible explanation for the spreading of 
cholera among the local population. Here it became clear that it was by 
working with those maps (see Figure 3.1) that Snow finally understood how 
the disease was transmitted. Barbara Tversky highlights this capacity of 
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visual representations as tools of thinking. She discusses Snow’s case and 
points out that it is often the case that “[iJn science [...] the underlying 
phenomena generating the data are not always known” (Tversky 2015, 
111). This was patently the case when Snow worked out his hypothesis by 
using maps. 

On a more general level, Tversky claims that it is often due to the ambi- 
guity of visual representations that they are such an effective means to develop 
problem solutions. She states that “[m]Jessy diagrams, then, can be crucial 
for thinking through problems, arousing and considering multiple possibil- 
ities. [...] Because ambiguity allows reconfiguration and reinterpretation, 
ambiguous sketches promote discoveries and inferences” (ibid.). Thus, what is 
commonly regarded as a shortcoming of visual representations, namely their 
apparent ambiguity, turns out to be particularly advantageous in the context 
of visual thinking.® 

Cases of visual thinking, then, suggest clear instances where images are 
inevitable tools to achieve understanding. Yet this doesn’t imply that images 
are indispensable in processes of achieving understanding more generally. It 
seems to be a mere contingent fact about the make-up of the reasoners’ minds 
that they rely on images in their cognitive processes. 

Secondly, there are cases where images are indeed indispensable to sci- 
entific understanding because they transmit information that cannot be 
expressed otherwise. At least some images are capable of transmitting non- 
propositional content, such as micrographs or depictions in guidebooks in 
biology. If this information is essential to understand the phenomenon at 
hand correctly, for example how to discern between different birds of prey 
during their flight, it seems that images providing this information can be 
called indispensable to the respective process of understanding. The example 
afforded by Perini is pertinent here. She points out that electron micrographs 
can “represent very complicated structural properties, even when there are 
no linguistic terms for the same features” (Perini 2005c, 921). That is, if the 
structures visible in such a micrograph are to be communicated, the utilisation 
of the image clearly becomes necessary since there are no proper terms avail- 
able to translate those features. Moreover, if this communicative exchange 
finally leads to an understanding of the reasons for the functions of those 
structures, it becomes clear in what sense certain images can indeed be indis- 
pensable to the process of achieving scientific understanding. 

As a concluding remark, we will discuss an objection to this claim put for- 
ward by Henk W. de Regt (see de Regt 2014). He agrees with the general idea 
that visual representations can facilitate scientific understanding. However, 
de Regt also claims that “visualization is a very effective way to achieve scien- 
tific understanding but it is not indispensable — there are other ways to reach 
the same goal” (ibid., 378, my italics). From his point of view, visualisability 
constitutes a theoretical quality that can unfold only in combination with par- 
ticular skills of the working scientists (see ibid., 380). Moreover, the acqui- 
sition of these skills is part of educational processes within a particular 
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community (see ibid., 393f.). In this sense, it is not a merely subjective char- 
acteristic of understanding but shared by members of a group. However, as a 
consequence of the necessity to learn depictive conventions beforehand, some 
scientists will not be able to profit from certain visualisations when trying 
to understand certain phenomena. They simply do not possess the relevant 
background knowledge to interpret them correctly, whereas others equipped 
with this information will benefit from the visual representation at hand. To 
illustrate his thesis, de Regt discusses the case of Feynman diagrams in physics 
(see ibid., 389ff.). 

The educational process that de Regt mentions in this context coincides 
with Fleck’s (and also Kuhn’s, as de Regt mentions (see ibid., 394)) ideas 
about scientific education. His approach is similar to theirs in that he also 
discusses the necessity to work with experienced scientists or practitioners to 
learn to see correctly, i.e. to make correct observations in their scientific fields 
(see Fleck 1979, 54, 104; Fleck 1986b, 118). 

De Regt is clearly right in pointing out that not all kinds of visual 
representations are equally intelligible to the human mind. The theory of 
image games allows us to acknowledge this fact without great difficulty. 
As there are varied people and varied intentions involved in such games, a 
layering of meanings can occur when visual representations are used in com- 
municative contexts. To understand those image games correctly, we then 
have to have the relevant background knowledge about the contexts in which 
those images are used. 

Moreover, it can be agreed that, as tools of communication, visual 
representations have been developed in the course of their application, and 
thus have been adapted to particular requirements of information transmis- 
sion — just as it is also the case with all other kinds of tools of communica- 
tion, for instance the usage of false colour photographs in astronomy. Here, 
scientists have to learn that those colours can, for example, indicate different 
altitudes (see Figure 2.12) or different kinds of radiation (see Figure 2.2). 
Thus, de Regt has a point in highlighting the fact that scientists have to acquire 
certain skills and background knowledge to interpret such images correctly. 

However, and contrary to de Regt, I think that, as image perception is 
just a special case of perception in general, it allows us to utilise the cog- 
nitive advantages provided by the evolutionary development of our visual 
apparatus. We are able to grasp non-conceptual content via images and 
we are particularly skilled in pattern detection. The example of the dis- 
covery of gravitational waves, discussed in the introduction to this book, 
is relevant at this point. Although the observer might not be familiar with 
the particular style of depiction of the diagram presented as evidence of 
this discovery (see Figure 1.2) and the theoretical assumptions supplying 
the background to this depiction, she might nonetheless be able to see the 
similarity between both curves. Thus, a kind of basic information seems to 
be transferable by visual means that does not presuppose prior education 
in interpretation. 
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This way of arguing, however, implies that people somehow deprived of the 
possibility of visually accessing the world, that is, in particular blind people, 
will not be able to acquire certain qualitative dimensions of non-conceptual 
information connected to the visual mode of accessing them. Information 
about colour and brightness might be paradigmatic examples here, whereas 
spatial information might also — at least partially — be accessed by tactile or 
aural means. 


4.3.4 Interim results: scientific images as a source of knowledge and 
understanding 


Whereas in the previous two sections our considerations have mainly 
concerned the possibility of transmitting and acquiring propositional know- 
ledge via using visual representations in scientific discourse, this last section is 
meant to broaden the focus. 

From an epistemological point of view, especially when recalling the 
prevailing sceptical attitude of traditional analytical philosophers towards 
this topic, that scientific images can indeed yield propositional knowledge 
under certain circumstances is a result of some interest. Nonetheless, such 
a conclusion still does not seem to be an entirely satisfactory answer to 
Perini’s question of why scientists use these images in the presentations of 
their results. If transmitting propositional knowledge were the only under- 
taking, linguistic expressions could clearly also be employed. Why therefore 
be concerned with visual representations? This laid out the framework in this 
last section to further analyse the cognitive value of scientific images. The 
question of whether additional merit is inherent in the practice of utilising 
visual representations in communicative contexts that cannot be dealt with by 
other kinds of representation or not is investigated at this juncture. 

The approach to this topic is based on the analysis of the process of 
learning as a paradigmatic cognitive process. A successful accomplishment 
of this cognitive undertaking yields two separate epistemic desiderata: know- 
ledge and understanding. Both are not only relevant in educational contexts, 
but also of major importance in scientific epistemic practices in general. The 
questions asked are whether and how visual representations can contribute 
to these epistemic achievements. Table 4.1 summarises the main results of the 
above analysis on this topic. 

The focus on the cognitive process of learning chosen for this analysis 
permits results obtained in empirical case studies in educational psychology 
and adjacent disciplines such as cognitive science and neuroscience to be taken 
into account. Amongst other things, it is made clear that certain constraints 
on the epistemic efficiency of images are a consequence of the recipients’ cog- 
nitive set-up. That is, the characteristics of students (for example, background 
knowledge, preferred form of information presentation, etc.) are relevant 
considerations, as are the challenges posed by the interplay between different 
representational means deployed to transmit information — the problem of 


322 The cognitive value 


Table 4.1 The cognitive value of visual representations in processes of learning 


Cognitive Potential cognitive Capacities of visual representations 
achievements achievements via visual enabling these achievements 
representations 
Knowledge Propositional knowledge Perceptual mode of information 
(knowing-that) access (presupposition: possession 
of relevant concepts) 
Practical knowledge Perceptual mode of information 
(knowing-how) transmission allows 
communication of visual 
demonstrations 
Phenomenal knowledge Transmission of non-conceptual 
/ knowledge by content supports mental images 
acquaintance 
Understanding Grasping of connections Perceptual mode of information 


between concepts and 
theories 


transmission can trigger 
retrieval of relevant background 


knowledge (dual coding theory) 


Perceptual mode of information 
transmission makes the display 
of relations and computational 
advantages possible (visual 
argument hypothesis) 


integration (see e.g. Renkl and Scheiter 2015, online first). Thus, the results 
obtained in educational psychology made particularly plain that it is not 
exclusively due to the kind of representational means deployed in the con- 
text of learning, but to other contextual matters as well, that the latter can be 
effective in the way intended. 

Moreover, the above discussion of two theoretical approaches, namely the 
dual coding theory and the visual argument hypothesis presented in detail by 
Vekiri (see Vekiri 2002), not only illustrates that visual representations can 
indeed have a positive impact on learning processes, but also reveals insights 
into how exactly they influence the recipients’ cognitive apparatus to enable 
this positive outcome. The main aspects are summarised in Table 4.1. 

At first sight, it might seem puzzling that the results obtained in educa- 
tional psychology are only attributed to the facilitation of understanding, but 
that this first impression might somehow be misleading is due to the following 
reasons. 

Firstly and admittedly, the capacity to offer information about indi- 
vidual elements and their relations, illustrated by proponents of the visual 
argument hypothesis, also affects the acquisition of knowledge, in par- 
ticular knowing-that. In this sense, their results are also of relevance to 
this epistemic achievement of learning. As these kinds of contribution of 
visual representations to knowledge acquisition are, however, implied by the 
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recipients’ perceptual way of accessing the encoded information, they are not 
mentioned separately in the table. A similar point has to be made about the 
possibility of storing visual information in the human brain, a thesis entailed 
in the dual coding theory. This clearly belongs to the capacity of images to 
transmit non-conceptual content. This is why it is not listed as a separate 
point. Furthermore, the remaining aspects of the processing of visual infor- 
mation and its cognitive merits seem to belong rather to the facilitation of 
understanding than to the mere amassing of units of information. 

Highlighting the fact that images often contribute to the achievement of 
scientific understanding does not imply a devaluation of the epistemic cap- 
acities of visual representations. This has been one of the main arguments 
presented above. On the contrary, the thesis was defended that this is one 
of the epistemic merits that make visual representations epistemically worth- 
while when used in scientific discourse. Philosophers such as Elgin, Kvanvig 
and Kosso are right in pointing out that understanding bears an intrinsic 
epistemic value. Examples show convincingly that something important is 
lacking if knowledge is acquired without understanding. Thus, giving sub- 
stance to the claim that visual representations can bring about this valuable 
epistemic desideratum, as the theses of educational psychologists based on 
and proven by empirical studies do, is of great importance to the project of 
correctly explaining the epistemic status of scientific images. 

The second epistemic merit resulting from the above analysis is related to 
the epistemic desideratum of knowledge. In addition to the capacity to convey 
propositional knowledge under certain circumstances, visual representations 
can also transmit practical knowledge, i.e. a kind of knowledge usually 
regarded as not communicable and, therefore, as epistemologically inferior 
to propositional knowledge (see Grundmann 2008, 86). Moreover, showing 
that visual representations can be used to transmit this kind of knowledge 
suggests that some standard of correctness can be applied to it. In this sense, 
the second objection mentioned by Grundmann, namely that knowing-how is 
not related to truth in any epistemologically relevant sense, can be called into 
question. Our rejoinder to this objection has been to show, by again drawing 
on results from educational psychology, that there are patently at least some 
paradigmatic instances of a correct and successful application of practical 
knowledge that can be transmitted via visual representations. One purpose 
of empirical studies in educational psychology also consists in an approach 
to these optimal ways of visually presenting certain information. Images are 
constantly subjected to redesigning processes to improve them for educational 
purposes. Thus, the standards of correctness suggested for practical know- 
ledge also bear on image design and its subsequent development in educa- 
tional contexts. 

Beyond that, it has been argued that using visual representations to transfer 
practical knowledge also considerably increases the number of recipients. 
Direct demonstrations of knowing-how usually only reach a small group 
of people. Making use of images, on the other hand, not only allows the 
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transmission of knowing-how on a synchronic level to a broader audience, 
but also on a diachronic level, as they can e.g. be published in textbooks or 
distributed in electronic learning courses and employed in students’ education 
for several years. 

Finally, an additional epistemic merit that visual representations can 
occasion is to provide recipients with concrete mental images. These 
images can be mentally manipulated to search for solutions to problems, as 
proponents of the visual argument hypothesis claim. Closely related to this 
point is also the fact that mental images play a significant role in that they 
enable counterfactual reasoning in science. Weighing and deciding between 
alternatives presupposes a clear understanding of the facts at hand — and, 
as has been shown above, sometimes this understanding cannot be achieved 
properly without visual representations that might also provide the scientists 
with a clear visual conception, i.e. a mental image, of the entity in question. 
Last but not least, Jung makes us aware that these images can also be used as 
tools for mental training to improve the deployment of skills obtained later 
(see Jung 2012, 181). 

All of these virtues are based on the fact that we make use of our visual 
perceptual apparatus to cognitively access information presented in scientific 
images. This way of deciphering the content allows us to draw on special skills 
related to vision, such as pattern detection, and we can gain access to non- 
conceptual content transmitted by visual means. 

None of the three epistemic achievements discussed above, that is, the 
facilitation of scientific understanding, the transmission of knowing-how and 
the conveyance of mental images, can be properly attained if we make use of 
other representational means in scientific discourse. These are then the three 
merits that answer Perini’s question about why it is that scientists use visual 
representations in the explanatory context of science. 


4.4 Summary 


In this chapter, the discussion has been on the question of the exact epistemic 
status of visual representations in the explanatory context of science. The 
investigation has focused on a comparative task: how the epistemic status of 
scientific images in comparison with other representational means — chiefly 
with linguistic expressions — is to be conceived. 

The starting point of the analysis was a detailed discussion of Perini’s 
theses on the topic. Her analytical approach to the subject matter at hand 
paved a way to revealing the diverse epistemic threads connected to the 
usage of visual representations in scientific discourse. One of the first aspects 
discussed in detail was the possible role of images in scientific arguments. 
Here, Perini insisted on their independent epistemic relevance. She made 
us aware of the fact that, although scientists treat visual representations as 
proper parts of scientific arguments, philosophers of science tend to neglect 
their epistemic value. 
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The next step was to get a better understanding of the nature of the 
philosophical problem lurking in the background and usually put forward 
as a rationale for this neglect in the philosophy of science. It became clear 
that at least two closely related aspects play a role in this. On the one hand, 
Steinbrenner points out that analytical philosophy is largely focused on lan- 
guage. On the other hand, this focus is still being passed on in the develop- 
ment of this philosophical school, in particular by maintaining Frege’s theses 
on thoughts — later called ‘propositions’ — as the truth-bearing contents of 
sentences that can be transmitted and be subjected to logical operations such 
as negations. Having initially set too narrow a focus on language might have 
tempted some philosophers to regard propositions themselves as language- 
like entities. However, even if we are not trapped within this narrow-minded 
framework, Frege’s argument that only propositions are proper candidates 
to bear truth values makes it rather difficult to maintain the claim that visual 
representations can make serious epistemic contributions in the context of 
scientific arguments. 

Perini’s solution to this problem is based on Goodman’s theory of depic- 
tion. His account constitutes the framework that she employs to clarify 
whether images are indeed bearers of truth values and therefore permis- 
sible components of scientific arguments. Earlier in this book, however, 
Goodman’s account has been rejected as a suitable approach to theorising 
about scientific images. Consequently, my solution diverges from the one 
suggested by Perini, although her analytical approach to the topic motivated 
my own reasoning. 

In the following, we analysed (a) whether visual representations possess 
a cognitive content of some kind, i.e. a content that can be transmitted via 
images in communicative contexts, and (b) we investigated the precise nature 
of such a presumed content. Maynard’s ideas on this topic were utilised as 
a guideline for this investigation. He emphasises that it is counterintuitive 
to point out, on the one hand, the intimate relation between the develop- 
ment of images and language in human prehistory and to argue, on the other, 
for an inferior status of images in comparison to language today. Maynard 
urges us to reconsider the question why it is that we willingly admit that lin- 
guistic expressions can transmit thoughts, but are reluctant to allow visual 
representations to fulfil the same epistemic task. 

Taking his challenge seriously, the suggestion was made to pay closer 
attention to the fact that our predominant way of accessing visually presented 
information is via perception. The question of what kind of information we 
are able to process cognitively via perception then seems to be related natur- 
ally to queries about the kinds of cognitive content transferable via scientific 
images. With this respect, two findings from epistemology were highlighted. 
On the one hand, contemporary philosophers usually regard perception as 
an epistemic source, yielding propositional knowledge. On the other hand, 
that we are able to perceive entities for which we do not have concepts, par- 
ticularly manifested by Hentschel’s example of the first observations of the 
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granulation of the solar surface, makes it plausible to claim that the con- 
tent of perception can be non-conceptual in kind. Contrary to Grundmann, 
who took this to support the thesis that it is only non-conceptual — and, 
thus, non-propositional — content that can be transmitted via percep- 
tion (see Grundmann 2008, 492f.), I argued for a more moderate claim, 
namely that the content of visual perception can be both propositional and 
non-propositional in kind. This seems to be possible because we acquire 
concepts by a mixture of testimony and perception. Initially we receive non- 
conceptual content via perception. This process is enriched in the course 
of our life by the concepts we acquire, which thus make the acquisition of 
propositional knowledge via perception possible, and constitutes a long- 
term development of our perceptual skills. 

The analyses presented in sections one and two of this chapter made 
plain that scientific images can indeed yield propositional knowledge. Yet 
inquiring into the perceptual mode of accessing information transmitted by 
visual means also revealed further merits that can be brought about when 
using visual representations in scientific discourse. By taking into account 
that our perceptual apparatus has been shaped by evolutionary processes, it 
can be called our predominant way of cognitively accessing the world. Here, 
not only were our abilities of pattern detection and colour differentiation 
pointed out, but also that we can acquire non-conceptual content by per- 
ceiving the world. 

The epistemic advantages of pattern detection for scientific purposes are 
immediately clear. Moreover, the human ability to grasp non-conceptual con- 
tent via perception also suggests another way in which visual representations 
can contribute to scientific arguments. Non-conceptual content can be particu- 
larly relevant when images are regarded as evidence of observational results. 
Here, another epistemic merit of scientific images became salient: they can 
support the detection of anomalies to prevalent scientific hypotheses, because 
they can entail content that is not already conceptualised in the vocabulary of 
the theory at hand. 

The epistemic merits employed by our perceptual apparatus to cognitively 
access visually presented information also laid out the framework for the 
last section of this chapter. Whereas the previous discussions suggested that 
some visual representations can indeed have parity with linguistic expressions 
when it comes to the transmission and acquisition of propositional know- 
ledge, we finally inquired into the possibility that sometimes scientific images 
can exhibit epistemic merits that cannot be achieved by using other represen- 
tational means. The discussion centred around the process of learning as a 
paradigmatic cognitive process, and that successfully accomplishing the pro- 
cess of learning yields both knowledge and understanding. 

The contributions that visual representations can make to bring 
about these two epistemic desiderata were examined. On the one hand, 
because of the capacity to transmit non-conceptual content and the per- 
ceptual mode to access visually presented information, scientific images 
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can supply recipients not only with propositional knowledge, but also with 
knowing-how and mental images. Thus, not only can some images transmit 
kinds of knowledge usually regarded as incommunicable (see Grundmann 
2008, 86), but they also allow us to bring the information thus gained to a cog- 
nitive effectiveness relevant in scientific reasoning, such as the manipulation 
of mental images to work on solutions to problems. On the other hand, by 
taking into account empirical results from educational psychology and adja- 
cent academic disciplines, it became comprehensible in what sense images can 
contribute to scientific understanding, defined as the grasping of connections 
between concepts and theories as propounded by Kosso and others (see 
Kosso 2007, 179). Moreover, by acknowledging that understanding is intrin- 
sically epistemically worthwhile, another important epistemic merit of visual 
representations, namely to facilitate understanding by being integrated into 
scientific discourse, was thereby revealed. 

To be concise, this analysis not only demonstrated how scientific images 
can contribute to scientific arguments and yield propositional knowledge, 
it also made clear in what sense at least some visual representations in cer- 
tain contexts can facilitate epistemic achievements that are not attainable 
via other representational means. Of course, only a claim about possibil- 
ities shall be defended here. Visual representations and also other vehicles 
of communication are subjected to certain constraints when realising 
their epistemic effectiveness. As we learnt from educational psychology, 
learner characteristics play an important role in this respect, as does the 
design of visual representations. Not all ways of visually presenting infor- 
mation are likewise appropriate to any epistemic context. Admittedly, as 
visual representations have become a tool of expressing human thoughts — 
‘thoughts’ here in a broad sense — they have also become subject to develop- 
mental processes of refinement and modification, as linguistic symbols have 
also. Visual skills have to be trained to decipher correctly the many different 
ways of visually presenting information. 

Having pointed out that visual representations can indeed be epistemically 
effective, and acknowledging the fact that they play an increasing role in 
science and society, some new responsibilities for scientists that appear in this 
constellation will be discussed briefly in the next chapter. 


Notes 


1 Taking ethics of science into account, we should add that it is not only epistemic 
practices that are subjected to such evaluations and improvements, but also 
practices that give rise to moral questions. 

2 It has to be added, however, that the considerations on justificatory reasons and 
practices, presented in the previous chapter, already exceeded a merely descrip- 
tive approach. We not only investigated how scientists do justify their epistemic 
practices, but also discussed the question of how they should justify their practices. 
Thus, the normative dimension was already inherent to our previous line of 
reasoning. 
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The model of information transmission used in the following discussion is based 
on related discussions in social epistemology, in particular in the debate about 
knowledge by testimony. For an introduction to the “transmission model of testi- 
mony” see Gelfert (2014, ch. 7). 

This focus on knowing-that in epistemology, however, has not stayed uncontested. 
Eva-Maria Jung, for example, tries to make a case for including also “practical 
forms of knowledge”, as she calls it, referring thereby to knowing-how (see Jung 
2012, ch. 1.5). 

This argument is presented in a similar fashion by Maynard (2009, sect. 2). 
Relevant discussions of the topic are especially connected to attempts at 
establishing a picture language, as mentioned in section 2.2 of this book. 

This is a translation of his paper Der Gedanke. Eine logische Untersuchung 
published in German in 1918/1919 (see Frege 1993, 30ff.). 

It has to be added that not all philosophers of science would agree that truth is the 
(ultimate) goal of science. Usually it is scientific realists who defend a claim along 
these lines, although most of them admit that other epistemic desiderata and also 
practical concerns might play a role. Thus, Ilkka Niiniluoto points out that there 
is no reason to assume that the cognitive aim of science has to be “one-dimen- 
sional”, that is, exclusively directed towards truth (see Niiniluoto 2015, ch. 2.4). 
Moreover, scientific anti-realists such as Bas C. van Fraassen explicitly deny that 
truth is the goal of science. Van Fraassen claims that scientists’ sole aim in their 
epistemic endeavours is “empirical adequacy” (see van Fraassen 1980, 12). 

He adds however that the captions of pictures can indeed be true or false, and 
discusses some interesting examples in this respect (see Gombrich 2004, 59ff.). 
Contrary to my suggestion, Richter accepts Goodman’s theory as the guiding 
model to make sense of the concept of scientific visualisations and tries to modify 
Perini’s work within this framework (see Richter 2014, ch. 4.2.2ff.). Having made 
my reasons sufficiently clear why I think that Goodman’s theory of depiction is 
not a suitable means to make use of in this context, Richter’s approach will be put 
aside without further discussion. 

In logics, arguments are usually considered to be purely linguistic in kind (see 
Salmon 1983). 

Meanwhile there is an extensive branch of studies flourishing on what might be 
called ‘diagrammatic reasoning’, some of them with an even more wide-ranging 
interest in diagrams and their role in epistemic reasoning than merely considering 
those mentioned above (see e.g. Giardino and Greenberg 2015; Shin, Lemon and 
Mumma 2014; Shin 2015; Stenning 2002; Tversky 2011 and references therein). 
Moreover, there are investigations into the possibilities of establishing completely 
visual logics (see e.g. Bagusche 2012). 

A fascinating example is Oliver Byrne’s book dedicated to the attempt to offer 
complete visual demonstrations and proofs of Euclid’s theorems — also meant as 
an attempt to introduce this complex material without using linguistic expression, 
and therefore supposed to be more easily comprehensible than a linguistic tractate 
on the topic might be (see Byrne 2013). Again, a lively debate also started about 
the relevance of visualisations in mathematics (see e.g. Brown 1996; Mancosu, 
Jorgensen and Pedersen 2005; Sazdanovic forthcoming and references therein). 
Richter mentions this as option (iii) of how images can play a role in scientific 
argumentation (see Richter 2014, 159). 

IUPAC is an abbreviation of ‘International Union of Pure and Applied Chemistry’. 
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Unfortunately, Scholz does not explain why he makes a distinction between 
contexts of usage and cultural contexts. It might be objected that cultural contexts 
are simply descriptions of contexts of usage. Ethnologists might offer explanations 
as to why we make use of certain images, and these explanations could then be 
called ‘cultural contexts’. 

Some of the ideas that are presented in this section are also discussed in M6Bner 
(2013b). 

Ana Laura Nettel and Georges Roque call this clear-cut distinction between 
persuasion and argumentation, between pathos and logos, into question. In 
particular, they doubt that the role of visual representations can exclusively be 
characterised as a kind of visual rhetoric. Contrary to this, they point out that 
although visualisations are often regarded as emotionally influencing, these 
representations can nevertheless play an argumentative part (see Nettel and 
Roque 2012, 67). 

This is also the way Vögtli and Ernst term the epistemic role of visual representations 
in argumentative contexts, in particular the role of visualisations produced by sci- 
entific instruments (see Vögtli and Ernst 2007, 36). 

A similar debate takes place in the philosophy of film (see Wartenberg 201 1b, 
12f.). Can what has been visually presented in a film be translated or paraphrased? 
One might argue that, as films are audiovisual media, the verbal argument can 
simply be peeled out of the mix. Yet this is not what the philosophy of film is 
about. Wartenberg makes this clear when stating that “[t]he crux of the debate 
is whether films within the standard genres of filmmaking — from fiction films to 
documentaries and even avant-garde films — can actually do more than raise a 
philosophical question or record a philosophical argument, whether some films 
should really be counted as doing philosophy on their own” (Wartenberg 201 1a, 
551). Proponents of film philosophy try to show that some films are capable of 
philosophising by filmic means only. That is, films are said to make significant 
contributions to philosophical debates. These contributions are transmitted in the 
audiovisual style of the medium itself and cannot be translated into other repre- 
sentational means. Stephen Mulhall, for example, defends this ambitious thesis 
(see Mulhall 2008). 

An overview is offered by McGrath (2014). 

Laurence BonJour (2013) offers an overview on different epistemological 
approaches to this topic in BonJour. 

For more information about the space mission see www.nasa.gov/mission_pages/ 
newhorizons/main/index.html, accessed May 24, 2016. 

The topic of understanding as an epistemic desideratum in science will be dealt 
with in more detail in section 4.3.3 of this book. 

Abstract paintings, for example, do not belong to this set of visual representations 
in Sober’s sense. Neither does he extend his theory to maps, diagrams, or graphs 
that mix different representational modes. Blair’s account is thus much broader 
in scope. 

This point is also highlighted by Dieter Mersch who argues that, although images 
can show entities (for example, people and their actions), they cannot show a nega- 
tive state of affairs (see HeBler and Mersch 2009, 19ff.). 

At least not without the aid of further symbolic auxiliary means, such as crossing 
out the content of a particular image to express that what it shows is not 
the case. 
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A discussion of recent approaches to this topic is provided by Gendler and 
Hawthorne (2009); Nanay (2014) and Schantz (2009). 

Scholz (2004) offers an insightful discussion of the common metaphor of epi- 
stemic sources. 

Philosophical sceptics have to be mentioned as an exception here, as some of them 
do not regard perception as a reliable source of information about the external 
world. However, scepticism seems to be a philosophical problem in its own right 
that needs not be restricted to particular epistemic sources, it being about the pos- 
sibility of knowledge in general. More information about how scepticism is related 
to perception is offered by Grundmann (2008, ch. 7.1). 

Eric Margolis and Stephen Laurence (2014) discuss different approaches to the 
concept of concepts. 

Further reasons are discussed by José Bermúdez and Arnon Cahen (2015, 
sect. 4.1). 

It has to be added that Peacocke does not speak about ‘representation’ in this 
context, as he takes this to involve figurative meaning. That is, the image 
of an aurochs in the Altamira cave can also be meant to represent success 
in hunting, not a particular animal. Such instances of a more wide-ranging 
meaning attribution via representation, Peacocke explicitly wants to exclude 
from his theory of depiction (see Peacocke 1987, 383). Thus, when speaking 
about ‘representational qualities’ in the following discussion of his ideas, 
I will use the concept in accordance with those constraints that result from his 
theoretical approach. 

Admittedly, this visual experience has to be repeated several times to learn the rele- 
vant characteristics. 

Artists, particularly fresco painters, are confronted with the problem of how to 
trick the human eye in a way that their paintings, situated on ceilings etc. and 
therefore often regarded from unfavourable angles, acquire their appropriate 
proportions nonetheless. Here it is the reverse, that is, spatial properties of the 
frescoes are willingly distorted so that their shapes appear without distortion in 
the recipient’s perceptual experience of them. Another example of tricking the eye 
in the domain of art is anamorphoses. 

With regard to this difficulty see in particular his explanations in Fleck (1986d). 
The topic of anomalies in observations is most famously discussed by Kuhn, who 
suggests that anomalies constitute bones of contention that will finally necessitate 
a shift in the prevalent paradigm (see Kuhn 1996, ch. 6). 

I have discussed this in more detail with regard to the training of journalists and 
their abilities to check their sources of information for accuracy. 

The following website offers access to more than 40 projects of this kind, see www. 
zooniverse.org, accessed January 11, 2016. 

What decisions have to be made by the user in her classificatory task is visualised 
in the decision tree produced by Coleman Krawezyk (University of Portsmouth), 
see https://data.galaxyzoo.org/gz_trees/gz_trees.html, accessed January 12, 2016. 
Marcel Boumans (2016) also discusses the relevance of human vision or, as he puts 
it, of the eye as “a reliable tool for judgment” in science. 

As an interesting spin-off, it can be added that this interrelationship between 
the indicative colouring of potential prey and the predators’ ability to perceive 
those colours triggered another evolutionary process which somehow aims at 
undermining the initial effect. Harmless potential prey have developed visual 
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characteristics of dangerous or poisonous species to lead their predators astray. 
Hoverflies, for example, look like wasps. 

Schawelka also discusses these phenomena, subsumed under the label of ‘reflect- 
ance’. He thereby refers to the ability of objects to reflect, absorb or transmit 
photons. Which of these abilities are realised depends on two aspects, namely 
(1) the structure or texture of the respective object and (2) the energy of the 
respective photon, i.e. the wavelength of the light (see Schawelka 2007, 45). 

Of course, it is not only in regard to colour perception that scientists have tried 
to overcome the limitations of the human eye, as the astrophysicist Thorsten 
Ratzka, for example, has pointed out in his retrospective description of human 
observations of celestial objects (see Ratzka 2012). 

His main examples are drawings, but he also applies his analytical results to other 
kinds of images, in particular to photographs and diagrams. 

An overview of different accounts in the context of argumentation theory is 
presented by Jens E. Kjeldsen (2015, ch. 2). 

Downes mentions this line of reasoning in his epistemological analysis of scien- 
tific images and attributes it to Pierre Duhem by stating that it was “[...] Pierre 
Duhem’s position that it was only human weakness that resulted in visual aids 
being required in the service of science” (Downes 2012, 117). Obviously, Downes 
hereby refers to Duhem’s theses on the two kinds of the human mind which the 
latter presents when discussing the relevance of models in science (see Duhem 
1998, ch. 4). From his point of view, it is due to a missing capacity of reasoning 
in abstract terms that some people need models or visual means of another kind 
to imagine the relevant relations in order to understand physical laws and the- 
ories correctly. 

I thank Johann Marek for helpful discussions of this topic. 

Some of the ideas presented in the following section have been published in 
MöBßner (2015). 

Of course, there may also be practical elements involved, for instance the prac- 
tising of certain skills to acquire the relevant expertise for deploying them later on. 
A multiplicity of ways to process and encode visual information mentally is also 
suggested by some of the case studies presented by the neuropsychologist Oliver 
Sacks. In “The Mind’s Eye” (Sacks 2010, 202ff.), for example, he discusses the 
medical histories of different people who went blind in the course of their lives. 
Interestingly, some of them kept their ability to construct mental images, that is 
to visualise objects, in their minds, whereas others totally lost this ability. These 
latter patients were nevertheless able to learn empirical facts about their envir- 
onments with the aid of their other senses. These medical case studies speak in 
favour of the thesis that there are at least two different cognitive subsystems for 
processing and encoding information, which can also be used separately if one 
of the systems is damaged or takes on new tasks from other parts of the brain. 
The advantage of visual representations to trigger visual imagings that can be 
further manipulated is also discussed by Max J. Kobert who offers some helpful 
examples (see Kobert 2010). 

Of course, there are innumerable design guides offering advice in this respect. In 
our previous discussion we have already learnt, for instance, about Frankel and 
DePace’s suggestions (2012). 

Michael P. Verdi and Raymond W. Kulhavy, for instance, mention gender as a rele- 
vant factor to consider when theorising about the educational efficiency of maps 
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(see Verdi and Kulhavy 2002, 33f.). Schnotz points out that the recipient’s age can 
play a role (see Schnotz 2002, 113). 

Regarding map comprehension, see e.g. Verdi and Kulhavy (2002, 33). 

Hausken’s and Roskies’s analyses of {MR images in medical practices show nicely 
what happens in interpretive processes if people (experts or laypeople) wrongly 
assume that they are familiar with the results of new imaging technologies (see 
Hausken 2015; 2017; Roskies 2007; 2008). 

Actually, he mentions three reasons. The possibility of cognitively processing 
knowledge can be regarded as a reason on its own. 

Jung discusses a variety of these strategies in detail (see Jung 2012, ch. 1.4.3.1 and 
1.4.3.2). 

Personal conversation with Max Hoffmann. 

There are different variants of this disease. Some can be so severe that the patient 
even loses her ability to understand linguistic expressions. 

A critical discussion of Jackson’s arguments, objections and rejoinders is offered 
by Martine Nida-Riimelin (2015). 

Again it has to be added that, in particular, the student’s background knowledge 
plays a major role with this respect. It might fill the gaps left open by the descrip- 
tion at hand so that a proper mental image can nonetheless be construed by the 
learner. A similar case can be made for images. The information transmitted 
by them might be as incomplete as the information transmitted by a linguistic 
description. Consequently, images serving educational purposes have to be care- 
fully chosen. 

The most famous ones are discussed by Michael Huemer (2011, sect. 3). 

For a critical discussion of such approaches see e.g. Koppelberg (1993). 

A compilation of important works on the topic of metaphors is presented in 
Ortony (1998). 

I discussed this point in more detail in M6Bner (201 3a). 

In this sense, images might be regarded as a kind of extended memory system, so 
to speak, though I do not want to relate this to the debate about the extended mind 
here. An introduction to this debate is offered by Holger Lyre (2010). 

Further examples of visual thinking in scientific and technological practices are 
discussed by Hentschel (2014, ch. 10). 


